STATUS  OF  DEPARTMENT  OF  DEFENSE  ARCHITECTURE 
FRAMEWORK  (DoDAF)  IMPLEMENTATION  WITHIN  THE 
AERONAUTICAL  SYSTEMS  CENTER  (ASC) 

THESIS 


Chad  A.  Millette,  Captain,  USAF 
AFIT/GSM/ENV/05M-03 

DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 

Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the  official 
policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense,  or  the  United 
States  Government. 


AFIT/GSM/ENV/05M-03 


STATUS  OF  DEPARTMENT  OF  DEFENSE  ARCHITECTURE  FRAMEWORK 
(DoDAF)  IMPLEMENTATION  WITHIN  THE  AERONAUTICAL  SYSTEMS 

CENTER  (ASC) 


THESIS 


Presented  to  the  Faculty 

Department  of  Systems  and  Engineering  Management 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
In  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Engineering  and  Environmental  Management 


Chad  A.  Millette,  BS 
Captain,  USAF 

March  2005 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT/GSM/ENV/05M-03 


STATUS  OF  DEPARTMENT  OF  DEFENSE  ARCHITECTURE  FRAMEWORK 
(DoDAF)  IMPLEMENTATION  WITHIN  THE  AERONAUTICAL  SYSTEMS 

CENTER  (ASC) 


Chad  A.  Millette,  BS 
Captain,  USAF 


Approved: 


/signed/  14  March  2005 

Ross  T.  McNutt,  Lt  Col,  USAF  (Chairman)  date 

/signed/  14  March  2005 

John  M.  Colombi,  Lt  Col,  USAF  (Member)  date 

/signed/  1 1  March  2005 


Bryan  J.  Hudgens,  Maj,  USAF  (Member) 


date 


AFIT/GSM/ENV/05M-03 


Abstract 

The  purpose  of  this  research  was  to  identify  the  current  status  of  the  use  of  the 
Department  of  Defense  Architecture  Framework  (DoDAF)  systems  architecture  products 
within  the  Aeronautical  Systems  Center  (ASC)  program  offices.  There  are  regulatory 
requirements  dictating  the  creation  of  DoDAF  products  as  annexes  to  programmatic 
documentation,  such  as  the  Joint  Capabilities  Integration  Development  System  (JCIDS) 
requirement  for  systems  architectures  as  annexes  for  acquisition  milestone  decision 
documentation.  In  addition,  the  DoDAF  itself  identifies  several  products  as  being  highly 
applicable  for  the  development  of  acquisition  strategies.  The  research  issue  was  to 
investigate  the  use  or  systems  architectures,  and  particularly  the  DoDAF  products,  within 
the  context  of  Air  Force  weapon  systems  acquisitions,  as  represented  by  ASC. 

The  research  indicated  two  conclusions:  while  programs  required  to  follow  the 
new  acquisition  processes  are  doing  so,  very  few  are  employing  systems  architectures 
systematically,  and  at  this  point,  at  least  within  ASC,  the  benefits  to  acquisition  program 
management  personnel  derived  from  an  architectural  context  are  not  yet  being  realized. 
These  conclusions  result  in  several  recommendations  to  ASC,  the  DoDAF  Working 
Group,  and  the  systems  engineering  community  in  general  as  to  how  to  make  systems 
architectures  more  a  way  of  doing  business  within  Air  Force  weapon  system  acquisitions 
efforts. 
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STATUS  OF  DEPARTMENT  OF  DEFENSE  ARCHITECTURE 


FRAMEWORK  (DoDAF)  IMPLEMENTATION  WITHIN  THE 
AERONAUTICAL  SYSTEMS  CENTER  (ASC) 

1.  Introduction 

1.1  Background 

In  February  2004,  the  Department  of  Defense  Architecture  Framework  (DoDAF) 
Volume  1  was  released  for  implementation.  This  was  actually  the  3rd  version  as  the 
framework  intended  for  all  DoD  systems  acquisitions  was  expanded  from  the  previously 
adopted  Command,  Control,  Communications,  and  Computer,  Intelligence,  Surveillance, 
and  Reconnaissance  (C4ISR)  Architecture  Framework  used  for  the  development  of 
software  intensive  applications  and  systems.  The  framework  was  part  of  a  larger 
Department  of  Defense  (DoD)  effort  to  reinvigorate  the  systems  engineering  process 
within  weapon  system  development  and  procurement.  The  framework  presents 
suggested  formats  for  the  modeling  of  systems  architecture  products  useful  at  various 
stages  of  the  weapon  systems  development  process. 

Systems  architecting  is  not  a  new  concept,  having  been  employed  in  the 
development  of  software  for  close  to  a  decade.  In  1996,  Hilliard,  Rice,  and  Schwann 
defined  architecture  as:  “the  highest  level  conception  of  a  system  in  its  environment”  in 
a  paper  attempting  to  expand  the  architecture  metaphor  beyond  the  software  development 
realm  into  the  general  systems  engineering  arena  (25:1).  In  2001,  the  Defense 
Acquisition  University  (DAU)  further  defined  system  architecture  as  “all  the  products 
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(including  the  enabling  products)  that  are  necessary  to  support  the  system  and,  by 
implication,  the  processes  necessary  for  development,  production/construction, 
deployment,  operations,  support,  disposal,  training,  and  verification”  (13:7). 

It  has  only  been  since  the  release  of  the  DoDAF  that  a  fonnal,  specific  framework 
has  been  in  place  for  the  development  of  systems  architectures  for  military  systems.  In 
addition  to  supporting  the  reemphasis  on  quality  systems  engineering  within  the  military, 
the  DoDAF  products  also  align  well  with  the  DoD  shift  to  capabilities-based  weapon 
system  development  and  the  specifically.  However,  there  is  always  a  temptation  with  a 
new  tool  to  attempt  to  make  it  all  things  to  all  people. 

1.2  Research  Problem 

Acquisition  professionals  deal  with  a  multitude  of  regulatory  requirements  when 
it  comes  to  managing  an  Air  Force  weapon  system  program.  Depending  on  the 
Acquisition  Category,  the  oversight  and  guidance  may  come  from  the  Under  Secretary  of 
the  Air  Force  for  Acquisition  (SAF/AQ).  With  respect  to  acquisition  program 
management,  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  (which 
replaced  the  previous  ‘Requirements  Generation  System’)  calls  for  architecture  views  as 
annexes  to  required  programmatic  documentation.  Further,  any  system  which 
communicates  with  other  systems  -  and  it  seems  hard  to  imagine  systems  being 
developed  in  today’s  net-centric  environment  that  do  not  have  a  requirement  to  interact 
with  other  systems  -  is  required  to  include  a  net-readiness  key  performance  parameter 
(NR-KPP)  as  one  of  its  requirements.  Finally,  the  DoDAF  itself  identifies  several  views 
for  program  managers  to  make  use  of  with  respect  to  acquisition  strategy  development. 
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In  terms  of  formal  policy,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 
(CJCSI)  3170.01,  Joint  Capabilities  Integration  and  Development  System  Instruction  and 
CJCSI  6212.01,  Interoperability  and  Supportability  of  National  Security  Systems,  and 
Information  Technology  Systems  directly  impact  program  managers  and  engineers  in 
program  offices.  CJCSI  3170  establishes  the  policies  and  procedures  for  “a  joint 
concepts-centric  capabilities  identification  process”  (8:1,2).  JCIDS  calls  for  specific 
DoDAF  architecture  views  as  annexes  to  documents  such  as  the  Initial  Capability 
Document  (ICD),  Capability  Development  Document  (CDD),  and  Capability  Production 
Document  (CPD)  required  at  each  milestone  of  the  development.  CJCSI  6212  “details  a 
methodology  to  develop  interoperability  Key  Performance  Parameters... based  on  the 
format  and  content  of  the  integrated  architecture  products  described  in  the  most  current 
version  of  the  DoDAF  (17:  2-7). 

In  addition,  the  Architecture  Framework  Working  Group  (AFWG)  identified 
eight  DoDAF  views  as  “highly  applicable”  to  the  development  of  a  successful  acquisition 
strategy  (17:3-12).  Although  not  specifically  addressed  in  policy,  these  views  are 
intended  to  serve  as  an  aid  to  the  program  manager  in  the  actual  management  of  the 
weapon  system  program.  System  program  office  (SPO)  personnel  are  deluged  with 
advice  and  guidance  on  how  best  to  successfully  manage  their  weapon  system  acquisition 
programs.  For  example,  “acquisition  Reform”  has  been  in  the  program  manager’s  lexicon 
for  several  years  now.  Further,  recently  the  emphasis  has  been  on  “Transformation”  in 
all  aspects  of  the  DoD,  with  certain  efforts  aimed  at  cycle-time  reduction  of  the  weapons 
systems  the  warfighter  requires.  How  are  systems  architectures,  and  the  DoDAF 
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specifically,  being  implemented  “in  the  trenches”  of  USAF  weapons  systems 
acquisitions? 

1.3  Research  Objective 

The  objective  of  this  study  is  to  examine  the  state  of  DoDAF  systems  architecting 
within  ASC  and  make  recommendations  to  improve  the  DoDAF  use  for  acquisition 
program  managers.  Specifically,  the  Defense  Architecture  Working  Group  is  in  the  early 
stages  of  developing  Version  2.0  of  the  DoDAF  and  should  benefit  from 
recommendations  resulting  from  an  analysis  of  the  current  implementation  effort  at  the 
program  office  level.  Further,  in  a  November  2004  presentation,  the  ASC  Chief 
Architect  made  the  following  assertion:  “(Wings/Direct  Reporting  Groups)  W/DRGs  are 
underway  in  developing  an  ‘architectural  understanding’  and  the  requisite  technologies 
for  new  net  enabled  capabilities”  (43:19).  Therefore,  as  a  result  of  this  study,  the  ASC 
Chief  Architect  will  have  better  insight  as  how  to  improve  DoDAF  implementation 
across  ASC. 

1.4  Thesis  Overview 

Chapter  2  will  provide  a  definition  of  systems  architecting,  a  discussion  of  the 
expected  benefits  of  the  DoDAF  for  acquisition  program  managers,  and  some  potential 
pitfalls  to  implementation  at  the  program  office  level.  The  DoDAF  is  not  the  first,  nor  is 
it  the  only,  systems  architecting  framework  and  Chapter  2  describes  two  others  as  well  as 
the  evolution  of  the  DoDAF.  The  next  section  introduces  the  DoDAF  views  and 
summarizes  the  expected  benefits  to  weapons  systems  acquisition  of  implementing  the 
DoDAF.  The  final  section  of  Chapter  2  provides  an  overview  of  previously  identified 
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obstacles  to  using  systems  architectures,  and  the  DoDAF  specifically,  for  product 
development.  Chapter  3  describes  the  data  collection  and  analysis  methodology  followed 
in  the  completion  of  this  inductive  study.  The  results  of  data  analysis  outlined  in  Chapter 
3  are  presented  in  the  Findings  and  Conclusions  sections  of  Chapter  4.  Finally, 
recommendations  for  the  AFWG  and  the  ASC  Chief  Architect  are  presented  as  well  as 
potential  areas  for  additional  research  in  Chapter  5. 
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2.  Literature  Review 


2.1  Overview 

As  discussed  in  the  previous  chapter,  the  publication  of  the  DoDAF  Version  1.0 
in  2004  was  not  the  first  foray  into  systems  architecting  as  part  of  the  overall  systems 
engineering  process.  This  chapter  provides  a  summary  of  the  relevant  literature 
pertaining  to  systems  architectures,  the  DoDAF,  and  Air  Force  weapon  systems 
applications.  The  systems  architecting  community  is  not  as  vast  as  one  might  think. 

There  are  recognized  experts  who  have  written  extensively  as  well  as  a  few  studies  that 
look  at  the  process,  products,  or  outcomes  of  systems  architecting.  The  wide  range  of 
literature  in  this  area  is  captured  below. 

First,  the  need  for  systems  architectures  as  part  of  the  overall  effort  to  reinvigorate 
systems  engineering  practices  within  DoD  weapon  systems  development  programs  is 
presented.  Following  this  introduction  are  several  definitions  and  descriptions  of  systems 
architectures  as  background.  A  brief  primer  on  the  Zachman  framework  and  the  IEEE 
Std  1471  is  also  presented  as  representative  of  other  architecture  frameworks.  This  leads 
to  the  evolution  of  the  DoDAF  from  C4ISR  Architecture  Framework  to  today  with 
descriptions  of  the  products  and  views  that  make  up  the  framework.  Next  is  a  discussion 
of  the  rationale  behind  using  systems  architectures,  and  particularly  the  DoDAF,  in  Air 
Force  weapons  systems  acquisitions,  presenting  both  general  as  well  as  regulatory 
guidance.  Finally,  recent  information  concerning  the  application  of  the  DoDAF  in 
weapons  systems  acquisitions  is  presented  with  an  emphasis  on  the  2003  Air  Force 
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Inspection  Agency  (AFIA)  Eagle  Look  report  on  architecture-based  acquisitions.  The 
Eagle  Look  report  will  lead  into  the  final  section  of  this  chapter,  which  deals  with 
roadblocks  or  issues  with  using  system  architectures  within  Air  Force  weapon  systems 
acquisition  efforts  that  have  been  previously  identified. 

2.2  Systems  Architectures  as  Part  of  Systems  Engineering 

In  February  2004,  Michael  Wynne,  acting  Under  Secretary  of  Defense  for 
Acquisition  Technology  and  Logistics  (AT&L),  issued  a  policy  letter  intended  to  begin 
the  process  of  reinvigorating  systems  engineering  (SE)  within  DoD  weapon  systems 
acquisitions.  Mr.  Wynne  stated  the  importance  of  rigorous  systems  engineering 
discipline  in  order  to  develop  and  maintain  needed  warfighting  capability.  Specifically, 
the  letter  called  for: 

All  programs  responding  to  a  capabilities  or  requirements 
document,  regardless  of  acquisition  category,  shall  apply 
a  robust  SE  approach  that  balances  total  system  performance 
and  total  ownership  costs  within  the  family-of-systems, 
systems-of-systems  context. 

He  went  on  to  say,  “collectively  these  actions  will  reinvigorate  our  acquisition 
community... thus  assuring  affordable,  supportable,  and  above  all,  capable  solutions  for 
the  warfighter”  (44:1). 

This  emphasis  on  systems  engineering  was  echoed  at  the  Air  Force  level  in 
comments  made  previously  by  both  the  Secretary  of  the  Air  Force  Dr.  James  Roche  and 
further  by  the  Secretary  of  the  Air  Force  for  Acquisition  Dr.  Marvin  Sambur.  In  a  24 
June  2002  Air  Force  Times  article,  Dr.  Roche  stated  in  response  to  questions  dealing  with 
issues  relating  to  recent  Air  Force  acquisition  program  budget  and  schedule  breaches, 


7 


“Increasingly,  I’m  convinced  that  the  systemic  problem  is  in  the  field  of  systems 
engineering”  (10:3  ).  In  his  9  April  2003  memo  “Incentivizing  Contractors  for  Better 
Systems  Engineering”,  Dr.  Sambur  said,  “An  immediate  transformation  imperative  for  all 
programs  is  to  focus  on  the  application  of  systems  engineering  principles  and  practices 
throughout  the  system  life  cycle”  (40:4).  One  systems  engineering  practice  looked  at  to 
help  in  this  reinvigoration  is  systems  architecting. 

An  underlying  rationale  behind  the  Air  Force’s  insistence  on  improved  systems 
engineering,  is  the  increasing  level  of  complexity  inherent  in  current  weapons  systems 
development,  an  issue  with  inherent  systems  engineering  implications.  Systems 
architectures  offer  a  tool  to  deal  with  this  issue.  In  a  systems  architecture  tutorial 
presented  at  the  2004  National  Defense  Industrial  Associates  (NDIA)  Systems 
Engineering  Conference,  presenters  from  Kasse  Initiatives,  LLC  stated:  “Generating  a 
system  architecture  as  part  of  the  systems  engineering  process  can  be  seen  as  a  deliberate 
approach  to  deal  with  the  uncertainty  that  characterizes  these  complex,  unprecedented 
systems”  (26:6).  Further,  Howard  Eisner  offers  the  following  in  his  book,  Essentials  of 
Project  and  Systems  Engineering  Management,  “Architecting  a  large-scale  complex 
system  is  the  centerpiece  of  systems  engineering”  (20:348).  Hilliard,  Rice,  and  Schwann 
go  so  far  as  to  offer  the  architectural  metaphor  as  an  appropriate  foundation  for  the 
systems  engineering  field  as  opposed  to  grounding  systems  engineering  in  other 
disciplines,  ranging  from  set  theory  to  systems  theory  to  category  theory  to  psychology 
(25:1). 

Whether  foundational  or  not,  “the  current  interest  in  architecture  is  motivated  by 
the  desire  to  build  our  systems  ‘faster,  better,  and  cheaper’”  (25:1).  “Faster,  better,  and 


cheaper”  has  been  a  weapons  systems  development  goal  for  years,  with  systems 
engineering  being  the  methodology  for  converting  requirements  into  systems  through  the 
DoD  acquisition  process.  “The  system  engineering  effort  is  integrated  into  the  systems 
acquisition  process  such  that  the  activities  associated  with  systems  engineering  support 
and  strengthen  the  acquisition  process”  (13:23).  Just  as  systems  engineering  integrates 
with  the  acquisition  process,  “systems  architecting  is  an  essential  part  of  the  system 
engineering  process  and  relies  on  many  of  the  methodologies  that  have  been  developed 
over  time”  (19:41).  As  with  any  entity,  multiple  perspectives  have  developed  over  time. 
These  perspectives  are,  in  some  way,  captured  in  the  multiple  definitions  of  systems 
architecture  in  the  literature. 

2.3  Systems  Architectures  and  Frameworks  Defined 

There  are  a  number  of  definitions  of  what  an  architecture  is  in  a  systems  context. 
Beyond  the  numerous  definitions,  there  are  several  frameworks  for  systems  architectures; 
these  include  the  Federal  Enterprise  Architecture  Framework,  The  Open  Group 
Architecture  Framework,  the  IEEE  1471  Standard,  the  Zachman  Framework,  and  the 
DoDAF.  Each  of  these  frameworks  has  its  own  definition  of  system  architecture  as  well 
in  addition  to  recommended  format  for  products.  The  similarities  and  distinctions 
between  these  different  definitions  are  worth  noting.  Irrespective  of  the  differences  in 
these  definitions,  the  bottom  line  is  that  for  Air  Force  weapon  system  acquisitions,  the 
definition  and  framework  that  applies  most  is  the  DoDAF. 

Beyond  the  three  frameworks  mentioned  above,  there  are  other  widely  accepted 
definitions  of  system  architectures.  Dr.  Mark  Maier,  author  of  The  Art  of  Systems 
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Architecting,  has  a  high-level  conception  of  systems  architectures  and  defines  the  process 
of  architecting  as  “the  art  and  science  of  developing  systems  solutions  in  ill-structured 
problem  environments”  (2:1).  Further,  he  believes  “the  concrete,  deliverable  products  of 
the  architect,  therefore,  are  models  (or  abstracted  designs)  of  the  system”  (33:18,139). 
This  high-level  perspective  is  shared  by  Hilliard  et.  ah,  who  stated:  “Systems  are  situated 
in  their  environments.  An  architecture  reflects  the  whole  system  in  response  to  that 
environment”  (25:1). 

NASA’s  definition  deals  with  functions  and  their  interactions:  “How  functions 

are  grouped  together  and  interact  with  each  other.  Applies  to  the  mission  and  to  both 

inter-  and  intra-system,  segment,  element,  and  subsystem”  (20:249).  The  Defense 

Acquisition  University  has  the  following  definition  that  seems  all-inclusive: 

The  System  Architecture  identifies  all  the  products 
(including  enabling  products)  that  are  necessary  to 
support  the  system  and,  by  implication,  the  processes 
necessary  for  development,  production/construction, 
deployment,  operations,  support,  disposal,  training,  and 
verification.  (13:7) 

Other  definitions  deal  with  architectures  role  in  the  design  of  the  system.  Howard 
Eisner,  author  of  Essentials  of  Project  and  Systems  Engineering  Management,  believes 
architecting  is  “fundamentally  a  design  or  synthesis  process”  and  defines  architecture  as 
“an  organized  top-down  selection  and  description  of  design  choices  for  all  the  important 
system  functions  and  subfunctions,  placed  in  a  context  to  assure  interoperability  and  the 
satisfaction  of  system  requirements”  (20:347,273).  In  his  book,  The  Engineering  Design 
of  Systems  Models  and  Methods,  Dennis  Buede  defines  an  operational  architecture  as 
providing 
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A  complete  description  of  the  system  design,  including 
the  functional  architecture  allocated  to  the  physical 
architecture,  derived  input/output,  technology  and  system- 
wide,  trade  off,  and  qualification  requirements  for  each 
component...  and  complete  documentation  of  the  design 
and  major  design  decisions.  (6:246) 


Finally,  Lawrence  McCaskill  believes  the  Federal  Chief  Information  Officer  Council’s 

definition  is  clearer  regarding  what  architectures  are,  and  their  intended  use: 

A  strategic  information  asset  base,  which  defines  the 
mission,  the  information  necessary  to  perfonn  the  mission 
and  the  technologies  necessary  to  perform  the  mission,  and 
the  transitional  processes  for  implementing  new 
technologies  in  response  to  the  changing  mission  needs. 

(37:3) 


These  definitions,  however  distinct,  all  present  an  architecture  as  a  representation 
of  a  system  that  facilitates  the  transition  from  user/customer  concept  to  actual  hardware 
or  software  implementation.  Whether  high-level  and  abstract  or  extremely  detailed  and 
technical,  the  point  is  still  the  same:  communicate  the  requirements,  design,  and 
constraints  involved  with  the  development  of  the  system.  Hilliard  et.  al.  sum  it  up:  “An 
effective  architecture  shows  how  to  build  a  system  to  satisfy  clients’  needs,  in  the  context 
of  that  client’s  goals  and  vision”  (25:3).  In  order  to  achieve  some  consensus,  frameworks 
have  been  developed  to  provide  some  structure  to  the  architecting  process. 

Three  frameworks  for  architectural  representation  include  the  Zachman  Enterprise 
Architecture  Framework,  the  IEEE  1471  Standard,  and  the  DoDAF.  John  Zachman 
created  and  published  a  Framework  for  Enterprise  Architecture  in  1987  and  extended  it 
for  broader  applications  in  1992  (45:5).  IEEE  Std  1471-2000  IEEE  Recommended 
Practice  for  Architectural  Description  of  Software-Intensive  Systems  was  published  in 
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October  2000  following  five  years  of  development.  And  “in  the  early  1990s  the  DoD 
undertook  the  development  of  an  architecture  framework  for  Command,  Control, 
Communications,  Computing,  Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR) 
systems”  (33:223)  which  has  evolved  into  the  DoDAF. 

The  Zachman  Framework. 

In  his  article,  “Architecture,  Enterprise  Architecture,  Frameworks,  and 
Processes”,  Kevin  Kreitman  describes  the  Zachman  framework  as  “perhaps  the  oldest 
and  most  extensive  framework  in  use  today”  (27:12).  The  Zachman  framework  consists 
of  six  categories  along  the  horizontal  axis  (data,  function,  network,  people,  time,  and 
motivation)  and  five  categories  along  the  vertical  axis  (scope,  business  model,  system 
model,  technology  model,  and  detailed  representations).  Although  designed  for 
enterprise  applications  such  as  reengineering,  David  Brown  wrote  in  the  Spring  2000 
Acquisition  Review  Quarterly  that  “the  Zachman  framework  provides  an  excellent 
template  for  developing  the  architecture  of  just  about  anything”  (5:125).  Further,  in 
Brown,  Zachman  defines  architecture  as  “that  set  of  design  artifacts,  or  descriptive 
representations,  that  are  relevant  for  describing  an  object  such  that  it  can  be  produced  to 
requirements  as  well  as  maintained  over  the  period  of  its  useful  life”  (5: 122).  Brown  also 
believes  “the  Zachman  framework  can  make  important  contributions  to  acquisition 
reform”  (5:125). 
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IEEE  Standard  1471. 


“In  April  1995  the  IEEE  Software  Engineering  Standards  Committee  (SESC) 

convened  an  Architecture  Planning  Group  (APG)  to  study  the  development  of  an 

architecture  standard  for  software-intensive  systems”.  Their  final  report  was  presented  in 

1996,  followed  by  the  IEEE  Architecture  Working  Group  holding  bi-monthly  meetings 

from  1996  to  1999  (24:4).  This  resulted  in  the  publication  of  IEEE  1471  Standard  1471  - 

2000,  Recommended  Practice  for  Architectural  Description  of  Software-Intensive 

Systems  in  2000.  “IEEE  1471  establishes  a  set  of  content  requirements  on  an 

architectural  description  -  a  collection  of  products  to  document  an  architecture”  (23:1). 

The  IEEE  Definition  of  architecture,  “the  highest  level  (essential,  unifying)  concept  of  a 

system  in  its  environment”  (22:4),  however  vague,  is  still  considered  by  many  the 

archetypical  definition.  Mark  Maier  offers  the  following  critique  of  the  1471  effort: 

The  1471  project  was  intended  to  codify  the  areas  of 
Community  consensus  on  architecture  description.  In 
the  end  ,  consensus  only  developed  around  a  framework 
of  views  and  viewpoints  and  an  organizing  structure  for 
architecture  descriptions,  but  there  was  no  prescription  of 
any  particular  views.  (33:230) 

Even  before  the  publication  of  the  standard,  Hilliard,  Rice,  and  Schwann  offered  a 
proposal  in  1996  to  extend  the  architectural  metaphor  beyond  software  engineering  to  the 
field  of  systems  engineering  in  general. 
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The  Evolution  of  the  DoDAF. 


It  is  this  broadening  of  perspective  that  characterizes  the  evolution  of  the  DoDAF 

from  a  software  and  C4ISR-intensive  system  framework  to  one  that  applies  now  to  all 

weapon  systems  development.  As  Maier  recounts: 

In  the  early  1990s  the  DoD  undertook  the  development  of 
an  architecture  framework  for  Command,  Control, 

Communications,  Computing,  Intelligence,  Surveillance, 
and  Reconnaissance  (C4ISR)  systems.  The  stated  goal  for 
this  project  was  to  improve  interoperability  across 
commands,  services,  and  agencies  by  standardizing  how 
architectures  of  C4ISR  systems  are  represented.  (33:223) 

The  Architecture  Working  Group  (AWG)  published  version  1.0  in  June  1996  and  version 

2.0  in  December  1997;  version  2.0  is  commonly  referred  to  as  the  C4ISR  Architecture 

Framework  (CAF)  (33:223-224).  An  early  1998  Joint  Staff  memorandum  mandated  the 

CAF  for  all  C4ISR  architecture  descriptions  (17:1-6). 

The  DoD  broadened  the  application  of  the  framework  beyond  C4ISR  systems 
based  on  the  utility  of  the  CAF  and  both  Federal  (Clinger-Cohen  Act  of  1996,  etc.)  and 
DoD  policy  encouraging  the  use  of  architectures  (17:1-6).  The  result  was  the  publication 
in  2004  of  the  DoDAF  Version  1.0  Volumes  I  and  II.  The  stated  purpose  of  the  DoDAF 
Version  1.0,  is  “to  provide  guidance  for  describing  architectures  for  both  warfighting 
operations  and  business  operations  and  processes”  (17:1-1).  DoDAF  Volume  I  defines 
architecture  as:  “the  structure  of  components,  their  relationships,  and  the  principles  and 
guidelines  governing  their  design  and  evolution  over  time”  (17:1-1).  Even  though  the 
ASC  Chief  Architect  defined  architecture  in  his  November  2004  presentation  on  Network 
Enabled  Warfare,  as  “a  systematic,  rigorous,  reproducible  methodology  for  capturing, 
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organizing  and  communicating  data  about  complex  systems  to  support  analysis”  (43:29), 


the  DoDAF  definition  is  the  one  used  and  implied  throughout  this  study. 


Description  of  the  DoDAF. 

The  DoDAF  consists  of  multiple  products  known  as  views.  There  are  four  types 
of  views,  the  All  Views,  Operational  Views,  Systems  Views,  and  Technical  Standards 
Views.  Several  of  these  views  are  collected  in  what  is  called  an  “integrated  architecture” 
referred  to  extensively  in  the  JCIDS  documentation.  These  are  the  architecture  products 
referred  to  throughout  this  research  effort. 

DoDAF  Volume  Two  defines  architecture  products  as: 

Those  graphical,  textual,  and  tabular  items  that  are 
developed  in  the  course  of  gathering  architecture  data, 
identifying  their  composition  into  related  architecture 
components  or  composites,  and  modeling  the 
relationships  among  those  composites  to  describe 
characteristics  pertinent  to  the  architecture’s  intended 
use.  (18,2004:1-1) 

Thus,  architecture  products  can  take  the  form  of  Power  Point  charts,  Excel  spreadsheets, 
tables  and  charts,  as  well  as  any  other  graphical  product  that  conforms  to  the  standard 
above.  The  DoDAF  is  careful  not  to  specify  a  certain  development  methodology.  In 
fact,  it  is  purposely  intended  to  be  methodology  independent  (12). 

There  are  four  categories  of  views  within  the  DoDAF:  the  Overview  and 

Summary,  Operational,  Systems,  and  Technical  Standards  Views.  The  All  Views 

category  captures  essential  overview  infonnation  about  the  architecture. 

The  Overview  and  Summary  (AV-1)  is  essential  for 
documenting  the  assumptions,  constraints,  and  limitations 
that  may  affect  high-level  decision  processes  involving. . . 
architecture.  AV-1  also  identifies  the  approving  authority, 
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the  completion  date,  and  records  level  of  effort  and  costs 
required  to  develop  the  architecture  as  well  as  the  time 
frame  covered  and  the  organizations  that  fall  within  the 
scope  of  the  architecture.  (17:3-10) 

“The  Operational  View  (OV)  describes  the  tasks  and  activities  necessary  to  successfully 
perform  the  mission,  the  participating  nodes,  and  the  associated  information  exchanges” 
(17:3-2).  Further,  “OV  descriptions  are  useful  for... defining  the  operational 
requirements  to  be  supported  by  resources  and  systems”  and  “a  pure  OV  is  materiel 
independent”  (17:3-2).  In  order  to  deliver  a  weapon  system,  the  tasks  and  activities 
modeled  in  the  OVs  are  allocated  to  systems,  which  are  themselves  modeled  in  Systems 
Views.  “The  Systems  View  (SV)  describes  the  systems  of  concern  and  the  connections 
among  those  systems  in  context  with  the  OV”  (17:3-3).  Finally,  “the  Technical 
Standards  View  (TV)  describes  a  profile  of  the  minimum  set  of  time -phased  standards 
and  rules  governing  the  implementation,  arrangement,  interaction,  and  interdependence 
of  systems”  (17:3-4).  The  DoDAF  defines  an  integrated  architecture  (a  tenn  used 
throughout  JCIDS  and  other  documents)  as  the  AV-1,  AV-2,  OV-2,  OV-3,  OV-5,  SV-1, 
and  TV-1,  at  a  minimum)  (17:1-5).  The  26  views  are  summarized  in  Figure  1. 
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Applicable 

View 

Framework 

Product 

Framework  Product  Name 

General  Description 

All  Views 

AV-1 

Overview  and  Summary 
Information 

Scope,  purpose,  intended  users,  environment  depicted, 
analytical  finding^ 

All  Views 

AV-2 

Integrated  Dictionary 

Architecture  data  repository  with  definitions  of  all  terms  used  in 
^j]l 2roduct^ 

Operational 

OV-1 

High-Level  Operational 

Concept  Graphic 

High-level  graphical/textua.  description  of  operational  concept 

Operational 

OV-2 

Operational  Node  Connectivity 
Description 

Operational  nodes,  connectivity,  and  information  exchange 
needlines  between  nodes 

Operational 

OV-3 

Operational  Information 
Exchanqe  Matrix 

Information  exchanged  between  nodes  and  the  re  evant 
attributes  of  that  exchanqe 

Operational 

OV-4 

Organizational  Relationships 
Chart 

Organizational,  role  or  other  relationships  among 
organizations 

Operational 

OV-5 

Operational  Activity  Model 

Capabilities,  operational  activities,  relationships  among 
activities,  inputs,  and  outputs;  overlays  can  show  cost, 
performing  nodes,  or  other  pertinent  information 

Operational 

OV-6a 

Operational  Rules  Mode 

One  of  three  products  used  to  describe  operational  activity— 
identifies  business  rules  that  constrain  operation 

Operational 

OV-6b 

Operational  State  Transition 
Description 

One  of  three  products  used  to  describe  operational  activity — 
identifies  business  process  responses  to  events 

Operational 

OV-6c 

Operational  Event-Trace 
Description 

One  of  three  products  used  to  describe  operational  activity — 
traces  actions  in  a  scenario  or  sequence  of  events 

Operational 

OV-7 

Logical  Data  Mode 

Documentation  of  the  system  data  requirements  and  structural 
business  process  rules  of  the  Operationa  View 

Systems 

SV-1 

Systems  Interace  Description 

Identification  of  systems  nodes,  systems,  and  system  items 
and  their  interconnections  within  and  between  nodes 

Systems 

SV-2 

Systems  Communications 
Description 

Systems  nodes  systems,  and  system  items,  and  their  related 
communications  lay-downs 

Systems 

SV-3 

Systems-Systems  Matrix 

Relationships  among  systems  in  a  given  architecture;  can  be 
designed  to  show  relationships  of  interest,  e  g.,  system-type 
interfaces,  panned  vs.  existinq  interfaces,  etc. 

Systems 

SV-4 

Systems  Functionality 
Description 

Functions  performed  by  systems  and  the  system  data  flows 
amonq  system  functions 

Systems 

SV-5 

Operational  Activity  to  Systems 
Function  Traceability  Matrix 

Mapping  of  systems  back  to  capabilities  or  of  system  functions 
back  to  operational  activities 

Systems 

SV-6 

Systems  Data  Exchange  Matrix 

Provides  details  of  system  data  elements  being  exchanged 
between  svstems  and  the  attributes  of  that  exchanqe 

Systems 

SV-7 

Systems  Performance 
Parameters  Matrix 

Performance  characteristics  of  Systems  View  elements  for  the 
appropriate  time  frame(s) 

Systems 

SV-8 

Systems  Evolution  Description 

Planned  incremental  steps  toward  migrating  a  suite  of  systems 
to  a  more  efficient  suite  or  toward  evolving  a  current  system  to 
a  future  implementation 

Systems 

SV-9 

Systems  Technology  Forecast 

Emerging  technologies  and  software.'hardware  products  that 
are  expected  to  be  available  in  a  given  set  of  time  frames  and 
that  will  affect  future  development  of  the  architecture 

Systems 

SV-lOa 

Systems  Rules  Model 

One  of  three  products  used  to  describe  system  fonctionality — 
identifies  constraints  that  are  imposed  on  systems  functionality 
due  to  some  aspect  of  svstems  desiqn  or  implementation 

Systems 

SV-1  Ob 

Systems  State  Transition 
Description 

One  of  three  products  used  to  describe  system  functionality — 
identifies  responses  of  a  system  to  events 

Systems 

SV-IOc 

Systems  Event-Trace 

Description 

One  of  three  products  used  to  describe  system  functionality — 
identifies  system-specific  refinements  of  critical  sequences  of 
events  described  in  the  Operational  View 

Systems 

SV-11 

Physical  Schema 

Physical  implementation  of  the  Logical  Data  Model  entities, 
e.g.,  message  formats  file  structures,  physical  schema 

Technical 

TV-1 

Technical  Standards  Proriie 

Listing  of  standards  that  apply  to  Systems  View  elements  in  a 
qiven  architecture 

Technical 

tv-2 

Technical  Standards  Forecast 

Description  of  emerging  standards  and  potential  impact  on 
current  Systems  View  elements,  within  a  set  of  time  frames 

Figure  1  DoDAF  Views  (DoDAF,  2004) 
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2.4  The  DoDAF  and  Air  Force  Weapon  System  Acquisition 

The  views  mentioned  above  are  intended  to  aid  the  weapon  system  designer  in 

translating  requirements  into  capability  to  the  warfighter.  Weapon  systems  design  and 

development  is  the  purview  of  program  managers  and  engineers  within  Air  Force 

acquisition  program  offices.  Beyond  the  notional  benefits  to  systems  engineering,  the 

DoDAF  Working  Group  prescribed  several  views  as  beneficial  to  the  program  manager 

in  acquisition  strategy  development.  Further,  as  Zinn  noted,  with  the  Clinger-Cohen  Act 

and  Office  of  Management  and  Budget  circular  A- 130,  “the  use  of  architectures  had  not 

only  been  recommended  but  essentially  made  law”  (46:17),  at  least  for  infonnation 

technology  systems.  In  addition,  JCIDS,  NR-KPP,  and  Infonnation  Support  Plan 

guidance  calls  for  the  production  of  systems  architecture  products  as  well.  These  are 

requirements  program  office  personnel  must  meet. 

The  most  basic  task  the  acquisition  program  manager  has,  albeit  far  from  a  trivial 

one,  is  to  translate  operational  requirements  into  a  contractual  specification  that  will 

result  in  the  development  of  a  system  meeting  the  user’s  needs.  This  is  the  core 

capability  systems  engineering  efforts  provide.  Systems  engineering  has  become  more 

and  more  complicated  as  the  level  of  complexity  of  the  systems  under  development 

increases  as  well  as  the  requirements  for  these  systems  to  interact  also  increase. 

Therefore,  “the  architectural  approach  is  needed  most  as  systems  become  more  complex 

and  multi-disciplinary,  and  for  systems  customized  to  individual  clients”  (3:1).  The 

DoDAF  Working  Group  stated: 

Using  an  integrated  architecture  ensures  that  the  system  to 
be  acquired  is  addressed  in  the  context  of  a  whole 
environment  rather  than  a  separate  entity.  The  architecture 
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can  support  identification  of  operational  dependencies 
outside  the  sphere  of  the  specific  system  under 
development.  (17:3-21) 

Further  In  a  2002  paper,  Dr,  Harry  Crisp  related  how  systems  architectures  can 
aid  in  this  effort,  stating:  “Architectures  provide  the  framework  for  FoS/SoS  (Federation 
of  Systems/Systems  of  Systems)  systems  engineering  and  acquisition”,  a  feeling  echoed 
by  Dr.  Steven  Long  (see  Figure  2  below)  as  well  as  the  Air  Force  Chief  Architect,  Dr. 
Alexander  Levis  (12:86).  This  belief  is  further  outlined  in  the  Architecture  Playbook 
developed  by  the  Enterprise  Integration  Forum  Architecture  Process  Team  as  a  guide  for 
the  use  of  systems  architectures:  “an  architecture-based  approach  can  provide  a  fonnal 
methodology  and  associated  language  for  determining  and  representing  similar 
information  about  complex  system  (system-of-systems)  and  relationship  to  their 
environments”  (19:1). 

Systems  architectures  are  another  tool  in  the  program  office  tool  box  that,  when 

combined  in  an  overall  management  and  execution  effort,  can  lead  to  success: 

Together,  integrated  architectures,  executable 
architectures,  analytical  tools  and  methods  render 
quantitative  actionable  information,  which,  in  turns 
supports  funding  decisions,  acquisitions,  system 
engineering,  and  investment  decisions.  (39:1 1) 

Architectures  are  also  a  tool  designed  to  aid  in  program  management.  Program  managers 

“need  to  be  able  to  analyze  these  architectures  to  locate,  identify,  and  resolve  definitions, 

properties,  facts,  constraints,  inferences,  and  issues  both  within  and  across  architectural 

boundaries  that  are  redundant,  conflicting,  missing,  and/or  obsolete”  (39:3).  In  fact,  this 

tool  can  be  considered  necessary,  “creating  a  system’s  C4ISR/DoDAF  architecture  is  one 

of  several  necessary  activities  to  advance  from  a  mission  concept  to  reality”  (32:10). 
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Figure  2  Architectures  as  Part  of  Development  Process  (Long,  2004) 


In  DoDAF  Volume  I,  the  DoDAF  Working  Group  identities  eight  views  as 

“highly  applicable”  to  the  development  of  a  weapon  system  acquisition  strategy. 

Systems  architecting  can  be  very  useful  in  this  early  stage  of  the  development  effort. 

The  role  of  systems  architecting  in  the  systems  acquisition 
process  depends  upon  the  phase  of  that  process.  It  is 
strongest  during  conceptualization  and  certification,  but 
never  absent.  Omitting  it  at  any  point,  as  with  any  part  of 
the  acquisition  process,  leads  to  predictable  errors  of 
omission  at  that  point  to  those  connected  with  it.  (33:23) 


Figure  3  depicts  the  “Recommended  Uses  for  Architectures”  as  identified  by  the  DoDAF 
Working  Group.  Notice,  under  ‘Acquisition  Process’,  the  first  line  is  Acquisition 
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Strategy.  There  are  eight  views  that  are  highlighted  as  “highly  applicable”  with  another 
three  as  “often  or  partially  applicable”.  In  addition  to  these,  Levis  describes  Dickerson 
and  Soules’  proposal  for  the  following  products  as  useful  for  acquisition  strategy 
development:  SV-8,  SV-9,  TV-2,  and  CV-6  (Capability  Views  never  implemented  in 
DoDAF)  (31:5-62). 
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Figure  3  Recommended  Uses  of  Architecture  (DoDAF,  2004) 


Figure  3  also  highlights  the  JCIDS  systems  architecture  requirements.  Notice  that 
the  views  under  the  JCIDS  header  are  not  only  “highly  applicable”,  but  are  also 
“specifically  addressed  in  policy”;  in  this  case  CJCSI  3170.  In  addition,  DoD  Instruction 
“Operation  of  the  Defense  Acquisition  System,”  May  12,  2003  “defines  how  integrated 
architectures  are  to  be  used  in  the  requirements  and  acquisition  processes”  (17:2-5). 
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“JCIDS  implements  a  capabilities-based  approach  that  better  leverages  the  expertise  of 
all  government  agencies,  industry  and  academia  to  identify  improvements  to  existing 
capabilities  and  to  develop  new  warfighting  capabilities”  (7:A-1).  In  fact,  the  assumption 
that  “integrated  architectures  are  the  preferred  method  for  describing  operational, 
technical  and  systems  interactions  and  assessing  future  capability  needs”  has  been  an 
underlying  theme  for  the  revised  JCIDS  documented  in  CJCSI  3170.01D  (35:3). 

Program  managers  attempt  to  get  their  system  through  the  acquisition  milestones 
to  full  production  and  sustainment.  In  order  to  accomplish  this,  they  are  required  to 
produce  the  appropriate  documentation  at  each  milestone  review.  “Integrated 
architecture  products  must  be  included  in  mandatory  appendixes  for  the  ICD,  CDD,  and 
CPD”  (17:2-7).  Further,  mandatory  integrated  architecture  products  for  CRDs  (Capstone 
Requirements  Documents)  include  AV-1,  OV-2,  OV-4,  OV-5,  OV-6C,  SV-4,  and  SV-6 
(8:E-A-6). 

In  addition  to  the  JCIDS  requirements,  architectures  are  also  required 
documentation  for  Information  Support  Plans  (ISPs  -  formerly  C4I  Support  Plans)  and  as 
part  of  the  documentation  required  to  identify  net-ready  key  performance  parameters 
(NR-KPP).  Both  Figures  3  and  4  (below)  identify  the  architecture  views  required  for 
ISP/C4ISP  development.  According  to  Chairman  Joint  Chiefs  of  Staff  Instruction 
(CJCSI)  6212. 01C,  “all  CDDs  (Capability  Development  Documents)  that  exchange 
information  will  have  a  NR-KPP”  which  is  “derived  from  a  completed  architecture  and 
developed  from”  mandatory  architecture  products  (see  Figure  4  below)  (9:F-1).  In  fact, 
the  instruction  goes  on  to  say  “development  of  the  NR-KPP  begins  with  designing  the 
architecture  for  the  proposed  system”  (9:F-2). 
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Figure  4  JCIDS  Documents/NR-KPP  Products  Matrix  (CJCSI,  2003) 


National  Security  Space  (NSS)  Acquisition  Policy  guidance  is  provided  in  NSS 
Acquisition  Policy  03-01  and,  although  separate  and  distinct  from  the  more  general 
weapon  systems  acquisition  guidance  in  the  DoD  5000  series,  emphasizes  the  use  of 
systems  architecture  products.  “It  is  the  responsibility  of  JCIDS  and  National  Security 
Space  Architect’s  (NSS A)  processes  to  develop  integrated  architectures  and  initial 
operational  view  (OV)  products  for  NSS  systems”  (42:10).  Further,  conducting  system 
architecture  development  efforts  and  producing  initial  SV  and  TV  architecture  products 
is  included  in  phase  readiness  review  and  entry  criteria  checklists  (42:  35).  Systems 
architectures  are  therefore  pervading  all  aspects  of  DoD  weapon  systems  development. 

In  a  presentation  before  the  Software  Technology  Council  in  2003,  Thilenius  and 
others  presented  the  following  statistic  highlighting  the  increased  role  architectures  are 
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playing  in  weapon  system  acquisition:  “Architecture  products  directly  responsible  for 
$1.17B  in  03  POM”  (41:26).  Therefore,  as  the  Air  Force,  and  indeed  all  of  DoD  moves 
to  a  more  capability-based  development  process,  systems  architectures  will  continue  to 
be  prevalent.  This  fact  is  highlighted  by  the  increased  interest  in  how  architectures 
should  integrate  in  the  weapon  system  acquisition  process. 

In  September  2003,  the  Air  Force  Inspection  Agency  published  its  findings  during 
an  Eagle  Look  investigation  into  architecture-based  acquisition.  Submitted  by  the 
Electronic  Systems  Center,  the  purpose  statement  was  to  “assess  the  ability  of  the  Air 
Force  to  integrate  enterprise  architecting  into  the  acquisition  process  by  identifying 
policy  strengths  and  shortfalls,  as  well  as  enablers  and  impediments  to  integration”  (2:no 
page).  The  Eagle  Look  team  interviewed  key  individuals  involved  with  architectures 
from  the  following  types  of  organizations  (113  interviews,  predominantly  senior  leaders 
-  70%  were  in  the  grade  of  Lt  Col  or  higher  for  military  and  GS-15  and  above  for 
government  civilians):  Secretary  of  the  Air  Force  and  Headquarters  United  States  Air 
Force  Functional  Offices,  Unified  and  Major  Commands,  Product  Centers  and  Product 
Groups,  and  Department  of  Defense  (DoD)  Functionals  and  Program  Offices.  The  team 
found  that  “94%  of  the  personnel  in,  or  involved  with,  the  acquisition  process  consider 
architectures  (both  warfighting  and  business)  to  be  of  significant  value  in  improving  how 
products  or  systems  are  acquired  and  sustained”  (2:no  page). 

2.5  Roadblocks  to  DoDAF  Implementation  Within  Air  Force  Product  Centers 

In  addition  to  the  positive  perceptions  with  respect  to  system  architectures  in 
acquisitions,  the  EAGLE  LOOK  team  also  identified  several  areas  of  concern.  There  are 
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others  experienced  in  systems  architectures  that  have  also  recognized  potential 
roadblocks  to  the  successful  implementation  of  systems  architectures  within  Air  Force 
weapon  systems  acquisitions.  At  the  2004  Command  and  Control  Research  and 
Technology  Symposium,  Lawrence  McCaskill  offered  his  analysis  of  the  DoDAF  and  its 
implementation.  Beyond  the  use  of  the  DoDAF  products,  there  are  concerns  about  the 
views  themselves.  One  of  the  recurring  critiques  of  systems  architectures  is  their  static 
nature,  leading  to  the  call  for  executable  architectures.  Finally,  there  is  a  danger  in 
program  office  personnel  creating  architectures  for  architecture  sake. 

A  FI  A  EAGLE  LOOK. 

Interviewees  responding  to  the  2003  AFIA  Eagle  Look  investigation  identified 
the  following  issues  to  be  addressed  in  order  to  move  to  an  acquisition  system  driven  by 
architectures  (see  Table  1). 
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Table  1  EAGLE  LOOK  Issues 


u 

Issues 

D 

Leadership  may  not  sustain  the  focus  needed  to  fully  implement  the  architecture 
construct. 

2. 

A  significant  portion  of  the  workforce  was  unconvinced  that  architectures  are  a 
valuable  construct  to  pursue. 

3. 

Policy  and  guidance  to  implement  an  architecture -based  acquisition  process  was 
insufficient. 

D 

Organizationally-  or  functionally-centric,  or  ‘stovepiped’  processes  will  impede 
the  move  to  an  enterprise  architecture-based  system. 

5. 

Full  integration  of  enterprise  architecting  into  key  Air  Force  and  DoD  processes, 
such  as  the  CRRA  (capabilities  review  and  risk  analysis)  and  the  Planning, 
Programming,  Budgeting,  and  Execution  (PPBE)  processes,  has  not  occurred. 

6. 

Air  Force  personnel  lacked  the  needed  education,  training,  and  experience  to 
effectively  pursue  enterprise  architecting. 

EB 

Funding  strategy  was  insufficient  to  accomplish  this  task 

(AFIA,  2003) 


In  addition,  “the  EAGLE  LOOK  team  identified  workforce  attitude  as  a  potential 
impediment  to  integrating  architecture  concepts  into  the  acquisition  business”,  which  is 
not  surprising  seeing  as  the  team  also  found  “the  Air  Force  does  not  include  architectural 
development  skills  as  a  core  skill  set  for  program  managers”  (2:16,  41). 

Many  EAGLE  LOOK  interviewees  felt  architectures  were  too  infonnation 
technology-centric,  with  one  respondent  stating  “(Architecture)  policy. .  .doesn’t  apply  to 
weapon  systems”  (2:16,  67).  This  latter  feeling  is  echoed  in  the  fact  that  the  only 
mention  of  systems  architecture  products  in  the  Interim  Guidance  Preceding  Air  Force 
Instruction  63-101,  Capabilities  Based  Acquisition  System  is  in  the  section  outlining 
information  technology  as  an  important  management  consideration  (14:20).  And  finally, 
with  respect  to  the  integrated  architectures  JCIDS  refers  to,  “none  of  the  overarching 
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joint  operational  architectures  are  in  place,  so  the  Services  continue  to  do  what  they’ve 
always  done”  (2:36). 

McCaskilVs  Study. 

Lawrence  McCaskill,  an  employee  of  Whitney,  Bradley,  &  Brown,  Inc., 
presented  a  paper,  "Integrated  DoD/C4ISR  Architectures:  It ’s  not  About  the 
Framework...  ”,  for  the  2004  Command  and  Control  Research  and  Technology 
Symposium  in  which  he  identified  several  concerns  with  the  way  architectures  were 
being  used.  His  study  set  out  to  “clarify  the  overarching  purpose  of  integrated 
architectures... and  describe  a  methodology  by  which  the  architecture  community  can 
improve  the  process  of  developing  and  maintaining  architectures”  (37:1). 

With  respect  to  the  CRRA  integration  issue  identified  in  the  EAGLE  LOOK 
report,  McCaskill  found  that  due  to  architectures  not  having  complete  financial  and 
scheduling  infonnation,  architecture -based  analysis  in  the  CRRA  “requires  lots  of 
manual  processes  to  put  together”  (37: 17).  Referring  to  acquisition  program  office 
personnel  reactions  to  the  requirement  for  architecture  products,  McCaskill  stated:  “This 
process  put  the  architectures  at  the  wrong  end  of  the  acquisition  chain;  the  architectures 
didn’t  drive  the  requirements  to  create  the  respective  systems  -  they  ended  up  being  the 
product  of  the  system  being  built  (and  often,  an  afterthought,  after  the  system  had  already 
been  built)”  (37:9).  Ultimately,  McCaskill  found  “current  efforts,  especially  with  regard 
to  C4ISPs/ISPs  are  ‘reinventing  the  wheel’  every  time  one  of  these  requirements 
documents  is  created,  thus  creating  semantic  mismatches  for  the  same  infonnation,  and  in 
the  endgame,  misusing  resources”  (37:16). 
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Issues  With  the  Products  Themselves. 


Despite  the  fact  that  the  DoDAF  has  been  embraced  by  the  DoD  as  the  best  tool 
to  add  discipline,  structure,  and  context  to  our  new  and  modernized  systems,  there  are 
some  issues  with  the  products  used  to  represent  the  architecture.  There  are  issues  with 
their  comprehensive  applicability  or  effectiveness  for  acquisition  management, 
capability-based  analysis,  and  systems  engineering  applications.  The  United  Kingdom 
(UK)  is  developing  its  own  architecture  framework,  in  part,  to  address  deficiencies  it  sees 
in  the  DoDAF.  Members  of  the  Joint  Staff,  specifically  the  J-8,  have  also  expressed 
doubts  concerning  the  utility  of  system  architecture  products  for  the  capability-based 
analysis  and  decision-making  they  perfonn.  Another  issue  with  the  products  themselves 
involves  the  ability  to  measure  the  level  of  functionality  or  capability  identified  in  the 
architecture. 

Hilliard  et.  al.  questions  how  complete  the  DoDAF  products  are  with  respect  to 

the  types  of  analysis  they  are  intended  to  aid. 

It  is  tempting  to  prescribe  predefined  views  (as  the 
DoDAF  does),  “however,  we  do  not  yet  have  enough 
experience  to  prescribe  these,  or  any,  views  for  all 
systems.  Sometimes,  a  system’s  most  critical  architectural 
concerns  fall  outside  this  familiar  set.  (25:4) 

Further,  Dam  and  Long  unearthed  another  issue  with  the  completeness  of  the  DoDAF 

architecture  products.  In  terms  of  comparison,  there  is  no  standard  set  of  levels;  for 

instance,  how  do  I  know  I  am  at  the  same  level  of  OV  when  looking  at  2  architectures 

(12)?  And,  in  terms  of  engineering,  Long,  Macdonald,  and  Maley  concur  with  the  notion 

the  DoDAF  is  less  than  complete,  believing  additional  detail  beyond  that  required  to 
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generate  the  views  is  needed.  “Otherwise,  the  team  performs  ‘engineering  by  view  graph’ 
which  is  well  known  as  an  inadequate  approach  for  the  design  of  complex  systems” 
(32:3). 


Maier  has  stated  the  following  in  terms  of  the  systems  engineering  aspects  of 

weapon  systems  acquisition: 

One  clear  issue  is  that  it  (CAF)  is  often  being  used  for 
purposes  for  which  it  was  not  intended.  The  goals,  at  least 
as  discussed  in  the  CAF  documentation,  did  not  include 
defining  a  standard  that  was  complete  with  respect  to  an 
acquirer’s  concerns.  For  example,  there  is  no  place  in  the 
views  for  performance  models,  cost  models,  or  other 
management  models.  Yet  all  those  are  clearly  necessary 
when  the  client  is  an  acquirer  and  must  make  acquisition 
decisions.  (33:226) 

With  respect  to  making  acquisition  decisions,  the  DoDAF  has  not  improved  upon  this 
deficiency.  As  such,  the  United  Kingdom  Ministry  of  Defense  (MoD)  is  working  on  its 
own  standard,  the  Ministry  of  Defense  Architecture  Framework  (MoDAF).  The  MoDAF 
builds  on  to  the  DoDAF  with  12  additional  views,  including  two  new  categories  of 
views;  Capability  and  Acquisition  Views.  These  additional  views  are  intended  “to 
handle  temporal  and  other  procurement  aspects  where  the  DoDAF. .  .doesn’t  cover  these 
aspects”  (36:  M-5).  The  ultimate  goal  is  to  develop  “a  common  language  set  to  describe 
systems  and  systems-of-systems”  and  obtain  a  “context  for  system  procurement”  not 
previously  achievable  (36:M-3).  Even  though  the  intent  of  the  DoDAF  is  to  bring 
interoperability,  consistency  and  cohesiveness  to  the  development  of  new  capabilities,  the 
products  from  these  tools  impose  an  additional  data  and  semantic  interface  that  the 
requirements  engineering  and  systems  engineering  teams  must  resolve”  (32:3). 
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In  a  17  May  2004  presentation  outlining  concerns  the  J-8  Staff  has  with  integrated 
architectures  US  Navy  Captain  Mike  Mara  declared  architectures  to  be  resource¬ 
intensive  and  not  as  valuable  as  hoped  as  planning  and  decision-making  aids.  He  said, 
“Architectures  tend  to  be  brittle,  static  at  best,  or  worse,  outdated.  DoDAF  architectures 
require  significant  fiscal  and  manpower  resources  to  produce”  and  “architectures  can  not 
be  produced  or  revised  on  a  timeline  that  matches  the  tempo  of  analytic  questions  facing 
decision  makers”  (35:4).  In  terms  of  utility  and  value,  Mara  said,  “Most  of  the  DODAF 
architecture  views  are  not  necessary  to  conduct  capability-based  assessments  and  do  not 
include  data  needed  to  support  this  process.  Additionally,  no  architecture  views  capture 
the  robust  relationships  between  capabilities,  attributes  and  metrics  needed  for  capability- 
based  assessments”  (35:4).  These  beliefs  have  lead  the  J-8  to  the  conclusion  that  they 
“no  longer  assert  that  architecture  is  the  preferred  method”  to  “evaluate  how  well  a 
system  or  system-of-systems  attains  a  desired  capability”  (35:5). 

An  additional  concern  with  the  DoDAF  products  is  their  inability  to  provide 

information  on  the  level  of  capability  or  effectiveness  of  the  architecture  modeled.  Mara 

also  recognized  this  deficiency  in  a  2003  paper, 

The  framework  only  show(s)  a  binary  relationship  between 
systems  and  operational  activities  -  a  system  either  has  the 
functionality  or  it  doesn’t.  In  reality,  there  needs  to  be 
recognition  and  some  assessment  of  how  much  functionality  a 
system  has”.  (34:9) 

McCaskill  also  noted  architecture’s  inability  to  provide  measures  of  effectiveness: 

While  this  answers  the  ‘first  order’  question  of ‘is  there  a 
system  being  developed  that  answers  the  requirements  of 
the  capability,’  it  does  not  answer  the  question  of  ‘how 
effective’  the  FoS/SoS  is  in  accomplishing  this  capability. 

Thus,  this  only  provides  the  ‘first  step’  towards  the 
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analysis  that  the  decision-maker  will  need  to  make 
acquisition  decisions”  (37:14). 

Executable  Architectures. 

One  way  to  answer  the  ‘how  effective’  question  McCaskill  brings  up  above  is  to 
run  an  executable  model  of  the  architecture  and  analyze  the  system/capability 
performance.  However,  the  DoDAF  does  not  include  an  executable  architecture  among 
its  products:  “There  are  currently  no  standards  for  the  format  or  process  for  constructing 
executable  architectures”  (17:  7-2).  Issues  of  timing  and  latency,  as  well  as  outcome 
measures  of  effectiveness  could  be  addressed  with  the  development  of  executable 
architectures. 

The  need  for  executable  architectures  lies  in  the  static  nature  of  the  DoDAF 
products.  “Static  operational  models  only  show  that  activities  ‘must  be  capable  of 
producing  and  consuming  information.  They  do  not  provide  details  on  how  or  under 
what  input/output  conditions  infonnation  is  actually  produced/consumed”  (39:  8).  In 
another  paper,  Ring  and  others  concluded:  “These  static  products. .  .fail  to  provide  a  good 
vehicle  for  conducting  detailed  dynamic  ‘behavioral’  analysis  of  how  the  systems  are 
supposed  to  interact  with  each  other”  (17:3).  James  Long  describes  the  need  for 
executable  models  as:  “Static  diagrams  may  or  may  not  actually  work,  since  in  reality 
many  of  the  processes  interact  with  one  another  and  functional  decomposition  can  miss 
critical  interfaces”  and  “simulation  enables  the  execution  of  these  models,  thus  ensuring 
that  the  design  is  executable  (i.e.,  will  work)”  (12:125).  And,  in  his  thesis  dealing  with 
the  implementation  of  a  specific  architecture,  Capt  Gregory  DeStefano  noted  that  despite 
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the  dynamic  behavior  captured  in  some  of  the  DoDAF  views  (in  particular  the  OV-6), 


“some  vehicle  is  needed  to  take  these  products  and  put  them  into  motion”  (15:2-14). 

Although  the  DoDAF  does  not  include  formats  or  processes  for  executable 
architectures,  it  does  define  what  they  are.  DoDAF  Volume  I  defines  executable 
architecture  as  “the  use  of  dynamic  simulation  software  to  evaluate  architectural  models” 
(17:7-1).  These  executable  architectures  would  provide  value  to  acquisition  program 
office  personnel,  as  stated  by  the  Enterprise  Integration  Forum  Architecture  Process 
Team: 


The  derivation  of  an  executable  model  of  the  architecture 
from  the  three  views  and  the  associated  integrated 
dictionary,  provides  a  basis  for  understanding  the 
interrelationships  among  the  various  architecture 
products  and  establishes  the  foundation  for 
implementing  a  process  for  assessing  and 
comparing  architecture.  (19:41) 


There  are  efforts  underway  to  address  this  issue  and  develop  or  improve  existing  discrete 
event  simulators  to  have  the  capability  to  perform  dynamic  analysis  of  the  capability  or 
system  under  development.  In  a  recent  Air  Force  Institute  of  Technology  thesis,  it  was 
shown  that  the  DoDAF  architectures  provide  all  the  infonnation  required  for  any 
modeling  and  simulation  required  to  analyze  competing  design  decisions  (46:91). 
However,  at  this  point  there  are  no  well  accepted  methods  for  a  program  office  engineer 
to  execute  an  architecture. 


Architecting  for  Architectures  Sake 

Irrespective  of  the  issues  with  systems  architecting  as  a  systems  engineering  tool 
or  with  the  DoDAF  and  its  views,  there  is  also  the  issue  that  the  architecture  becomes  the 
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focus,  as  evidenced  in  the  following  caution  from  Jeffrey  Harris,  former  Director  of  the 


National  Reconnaissance  Office: 

Recent  years  have  increased  the  focus  on  architectures. . . 
While  (this)  is  beneficial,  it  is  easy  to  allow  the  architecture 
and  its  processes  to  become  the  focus  rather  than  the  users’ 
desired  effects”.  (2 1 :47) 


The  DoDAF  Deskbook  attempts  to  head  off  this  issue  by  providing  a  notional  six-step 
process  that  includes  four  steps  before  any  architecture  products  are  actually  built  (see 
Figure  5  below). 


Figure  5  Architecture  Development  Process  (DoDAF  Deskbook) 


Further,  McCaskill  also  cautions  that  “while  the  Framework  plays  a  large  part  in 


providing  a  common  lexicon  by  which  the  primitives  that  compose  integrated 
architectures  are  described,  delving  directly  into  ‘spreadsheets  and  boxologies’  misses 


the  point  of  why  we’re  creating  integrated  architectures”  (37:3).  It  would  be  a  shame  for 
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those  earnestly  working  on  systems  architectures  in  an  effort  to  improve  their  systems 
engineering  rigor,  to  lose  focus  on  the  reason  for  the  system  architecture  and  indeed  the 
systems  engineering  work  at  all. 

System  architecture  products  are  required  by  direction  for  Air  Force  weapon 
systems  acquisition  efforts.  The  JCIDS  process  identifies  capability  requirements  that 
evolve  into  the  weapon  systems  that  are  developed  in  Air  Force  Materiel  Command 
product  centers.  These  product  centers  employ  program  managers  and  engineers  skilled 
in  taking  a  user’s  requirement  and  turning  it  into  a  hardware  or  software  solutions,  that  is 
systems  engineering.  The  DoDAF  provides  a  framework  for  capturing  the  early  systems 
engineering  work  performed  and  communicating  the  complex  interactions  of  the  system 
or  capability  under  development.  However,  the  DoDAF  has  not  yet  been  universally 
accepted  as  a  systems  engineering  tool  and  practice.  There  are  several  obstacles  to  the 
full  implementation  of  a  systems  architecture  mentality  within  the  trenches  of  Air  Force 
weapon  systems  acquisition. 
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3.  Data  Collection  and  Analysis  Methodology 

3.1  Overview 

Chapter  2  summarized  the  literature  supporting  the  notion  that  systems 
architectures  are  useful  and  valuable  for  Air  Force  weapon  systems  acquisitions. 
However,  there  were  also  several  issues  or  roadblocks  identified.  Chapter  3  lays  out  the 
methodology  for  investigating  the  level  to  which  these  obstacles  have  been  overcome  (at 
least  within  ASC).  This  study  is  inductive  in  nature,  relying  on  interviews  with  “people 
in  the  know”  and  critical  analysis  of  the  results.  First,  the  “people  in  the  know”  had  to  be 
identified  -  the  population  from  which  to  collect  data.  Then,  the  data  was  collected  via 
structured  interviews.  This  data  was  collated  and  formatted  for  analysis  of  the  results. 
This  analysis  involved  grouping  respondents  by  their  level  of  DoDAF  implementation. 
Along  the  way,  significant  additional  information  concerning  systems  architecting,  the 
DoDAF,  capabilities-based  system  development,  and  even  the  new  ASC  organizational 
structure  was  also  collected. 

3.2  Research  Design 

This  is  a  qualitative  case  study  of  the  implementation  of  system  architectures, 
specifically  the  DoDAF,  within  ASC.  Whereas  the  2003  AFIA  Eagle  Look,  described  in 
Chapter  2,  took  a  broad  brush  look  at  architectures  in  acquisition  across  the  Air  Force  at  a 
senior  leader  level,  the  focus  here  is  collecting  detailed  information  to  determine  the  level 
to  which  the  DoDAF  has  permeated  throughout  ASC  to  the  program  manager  and 
engineer  level.  Data  was  collected  through  interviews  (and  in  rare  cases  e-mail 
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correspondence)  with  personnel  representing  the  highest  level  of  acquisition  program 
management  execution.  These  interviews  lead  to  follow-up  interviews  with  specific 
program  personnel  within  each  Wing  or  DRG.  In  total,  39  interviews  were  conducted. 

3.3  Data  Collection  Methodology 

The  data  collected  through  these  interviews  was  intended  to  provide  a 
characterization  of  the  level  of  system  architecture  work  previously  done,  currently 
ongoing,  or  planned.  The  questions  posed  during  the  interviews  gathered  infonnation  as 
to  rationales  behind  decisions  concerning  architecture  efforts  within  each  organization. 

In  addition,  data  was  collected  concerning  the  level  of  education,  training,  and  general 
familiarity  in  systems  architectures  each  person  interviewed  possessed.  In  total,  this  data 
would  be  collated  by  program  and  then  by  Wing  or  DRG  in  order  to  portray  an  overall 
picture  of  systems  architecting  within  ASC  as  a  whole. 

The  interviews  generally  took  no  more  than  30  minutes  and  were  facilitated  by  a 
standard  note-taker  template.  Access  to  the  interviewees  was  made  possible  through  the 
ASC  Commander’s  Action  Group  and  through  personal  connections  with  the  Wing  and 
DRG  Executive  Officers.  Each  interviewee  was  assured  that  their  individual  identities 
would  not  be  revealed.  Confidentiality  is  accomplished  by  reporting  the  responses  tied  to 
organizations  rather  than  specific  persons  or  even  job  titles. 

Although  no  two  interviews  were  exactly  the  same,  the  same  basic  questions  were 
asked  to  all  interviewees.  Example  questions  included: 
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-  Does  your  organization  (Wing/DRG/Program,  depending  on  the  level  of 
authority  of  the  person  being  interviewed)  make  use  of  the  DoDAF  systems 
architecture  framework/process? 

-  If  yes; 

—  To  what  degree  do  you  make  use  of  the  architecture  products  during  the 
execution  and  management  of  your  program? 

—  Which  views  are  more/less  useful  to  you  in  executing  your  program? 

—  Who  creates  the  views;  in-house  (program  office  personnel  and 
contractors),  outside  contractor  support,  the  user  (Major  Command)? 

—  How  much  training  or  education  do  the  people  creating  and  reviewing 
the  architecture  views  have  in  the  DoDAF  and  systems  architecting,  in 
general? 

—  What  tools  do  you  use  to  create  the  systems  architecture  views  (Popkin, 
Visio,  Power  Point,  etc.)? 

-  If  no; 

—  Why  not?  What  is  keeping  your  organization  from  adopting  this 
framework? 

—  Does  your  organization  employ  another  systems  architecture 
framework  (IEEE  1471,  Zachman,  etc.)? 

—  Do  you  have  another  method  for  capturing  the  output  of  your  systems 
engineering  processes? 

-  How  could  systems  architecting,  and  specifically  the  DoDAF,  be  more  useful  to 
you? 
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The  population  for  this  study  was  selected  in  order  to  best  meet  the  research 
objective.  ASC  provided  a  population  experiencing  the  expansion  of  the  role  of  the 
DoDAF.  The  DoDAF  is  expanding  the  realm  of  systems  architecting  from  C4ISR 
systems  to  all  systems.  As  such,  studying  the  implementation  at  the  Aeronautical 
Systems  Center  seemed  appropriate.  The  level  of  acceptance  and  use  within  ASC  should 
serve  as  a  barometer  for  overall  acceptance  of  the  DoDAF  within  Air  Force  weapon 
systems  acquisition  programs,  which  traditionally  have  not  been  dominated  by  networked 
C4ISR  capabilities. 

ASC  was  undergoing  a  significant  reorganization.  In  June  2004,  ASC  began 
operating  under  a  wing,  group,  and  squadron  structure.  Over  40  separate  program  offices 
were  organized  into  five  acquisition  systems  wings  and  two  direct  reporting  groups  for 
fighter  attack,  long  range  strike,  reconnaissance,  mobility,  agile  combat  support,  special 
operations  forces  and  training  aircraft.  In  January  2005,  the  structure  was  formally 
recognized  (1:2).  The  new  structure  is  depicted  in  Figure  6.  ASC  was  the  first  product 
center  to  operationalize  this  reorganization  with  all  the  others  soon  to  follow. 


38 


Figure  6  ASC  Reorganization  Chart  (ASC,  2005) 


Within  ASC,  interviews  were  perfonned  at  various  levels.  The  first  interviews 
were  with  members  of  the  ASC  Commander’s  Action  Group  (CAG).  The  CAG  is 
responsible  for  being  the  liaison  between  the  Wings  and  DRGs  and  the  Commander. 
They  review  documentation  and  generally  make  sure  both  the  Commander  is  infonned 
about  the  Wings/DRGs  and  the  Wings/D RGs  are  informed  about  the  concerns  of  the 
Commander.  The  CAG  was  chose  first  in  order  to  get  an  overall  picture  of  the 
Commander’s  requirements  with  respect  to  systems  architecture.  For  instance,  does  the 
Commander  review  the  systems  architecture  products  included  in  the  programmatic 
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documentation  that  flows  from  the  programs?  Also,  the  CAG  provided  connections 
within  the  Wings  and  DRGs  for  further  interviews. 

Within  each  Wing/DRG,  the  first  interviews  were  conducted  with  senior  program 
office  personnel  with  program  management  and  engineering  responsibilities.  These 
leaders  were  a  mix  of  active  duty  Air  Force  officers  (Majors,  Lieutenant  Colonels,  and 
Colonels)  and  government  civilians  (GS-13,  14,  15,  and  SES).  These  interviews  were 
intended  to  provide  an  overview  of  the  systems  architecture  work  ongoing  within  the 
organization  as  a  whole  as  well  as  the  senior  leader  perspective  on  systems  architecting  in 
general.  When  systems  architecting  work  was  indicated  within  specific  programs  within 
the  organization,  follow-up  interviews  were  scheduled  with  the  appropriate  programmatic 
personnel. 

The  follow-up  interviews  concentrated  on  the  specific  program  application  of 
systems  architecting.  Interviews  were  conducted  with  program  managers,  engineers,  and 
other  program  office  personnel.  These  interviewees  included  a  wide  range  of  active  duty 
Air  Force  officers,  government  civilians,  and  support  contractors.  The  focus  of  this 
sampling  group  was  an  in-depth  review  of  the  program  status  and  any  system  architecture 
work  ongoing  or  planned.  This  group  was  also  best  suited  to  respond  to  questions 
dealing  with  education,  training,  and  experience  as  they  represent  the  “hands-on” 
architecture  workforce. 

Within  ASC,  three  other  groups  were  also  interviewed  in  order  to  broaden  the 
scope  of  the  study  to  encompass  the  entire  organization.  First,  as  part  of  the  ASC 
reorganization  a  Capabilities  Planning  and  Integration  Directorate,  ASC/XR  was  stood 
up.  This  organization  is  intended  to  be  the  origin  of  new  system  development  efforts 
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within  ASC  by  coordinating  the  user’s  initial  capability  requirements  with  the 
appropriate  Wing  or  DRG.  Since  this  group  has  a  program  initiation  function,  they  made 
likely  candidates  for  architecture  work.  The  next  two  groups  are  related  in  their  support 
function  within  ASC.  ASC/PM  provides  education  and  career  management  support  to 
the  program  managers  assigned  to  ASC,  while  ASC/EN  performs  the  same  function  for 
the  engineering  personnel.  Both  of  these  organizations  had  the  potential  to  effect 
architectural  implementation  through  their  policy  and  standards  and  education  roles.  The 
personnel  interviewed  in  these  organizations  were  government  civilians  in  the  grades  of 
GS-13  and  above. 

Additionally,  interviews  were  conducted  with  systems  architecture  and  DoDAF 
subject  matter  experts.  In  the  course  of  reviewing  the  relevant  literature  with  respect  to 
systems  architectures  and  the  DoDAF,  these  experts’  names  and  contact  information 
became  available  as  resources  for  data  collection.  The  three  experts  interviewed  were  all 
government  support  contractors  with  vast  experience  in  either  systems  architecting  and/or 
the  DoDAF,  in  particular.  These  interviews  (all  conducted  via  e-mail)  focused  on 
gathering  expert  opinion  on  the  current  issues  surrounding  program  office 
implementation  of  the  DoDAF  systems  architecture  framework. 

3.4  Data  Analysis  Methodology 

This  study  involves  an  interpretational  analysis  methodology.  In  Leedy,  Gall  et. 
al.  describe  this  process  as  “examining  the  data  for  constructs,  themes,  and  patterns  that 
can  be  used  to  describe  and  explain  the  phenomenon  studied”  (30:158).  The 
phenomenon,  in  this  case  is  the  implementation  of  the  DoDAF  system  architecting 
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framework  within  ASC.  The  data  collected  provides  information  enabling  “an  inductive 
process  of  organizing  data  into  categories  and  identifying  patterns  (relationships)  among 
the  categories”  (McMillan  &  Schumacher  quoted  in  30:  165).  Combining  these  ideas, 
the  analysis  methodology  for  this  study  involves  grouping  like  organizations  in  a  three¬ 
tiered  scale  in  terms  of  level  of  architectural  implementation  and  identifying  overarching 
patterns  of  behavior  with  respect  to  systems  architecting.  Representative  quotes  from 
interviewees  were  collected  in  order  to  support  the  characterization  of  each  organization. 

The  interviews  provide  data  with  which  to  characterize  each  program,  DRG,  or 
Wing  with  respect  to  the  level  of  systems  architecting  implementation.  The  organizations 
can  then  be  plotted  on  a  continuum  representing  various  levels  of  systems  architecture 
implementation.  A  generic  continuum  is  presented  in  Figure  7.  This  continuum  will 
provide  a  graphical  depiction  of  the  number  of  organizations  in  each  stage  of  systems 
architecture  implementation.  This  depiction,  along  with  selected  quotes  and  other  data  to 
clarify  each  organization’s  placement  on  the  continuum,  should  prove  and  could  provide 
a  baseline  for  further  ASC  architecture  implementation  as  well  as  background  for  the 
development  of  version  2  of  the  DoDAF. 
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Figure  7  Architectural  Implementation  Continuum 
The  second  portion  of  the  analysis  involves  recognizing  the  overall  themes  and 

patterns  of  systems  architecture  behavior  within  ASC.  This  data  is  culled  from  questions 
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dealing  with  the  perceived  value  of  systems  architectures  to  acquisition  program 
management  as  well  as  recommendations  suggested  by  interviewees.  The  results  of  this 
analysis  will  be  presented  in  verbal  form  with  representative  quotes  from  the 
interviewees. 

3.5  Validity  and  Reliability 

In  a  qualitative  study  such  as  this,  there  appears  to  be  no  “single,  commonly 
accepted  standard  forjudging  the  validity  and  reliability”  (30:168),  however  this  does  not 
lessen  the  concerns  with  respect  to  ensuring  these  aspects  of  the  study  are  maintained  at  a 
high  level.  Validity  deals  with  the  effectiveness  of  the  measure;  does  it  measure  what  it 
is  supposed  to  measure?  How  comprehensively?  How  accurately?  (30:32).  Reliability 
deals  with  the  consistency  with  which  a  measurement  perfonns.  Does  the  instrument 
consistently  measure  the  factors  it  was  designed  to?  Does  it  do  so  accurately  (30:35)? 
Both  issues  are  addressed  below. 

In  Leedy,  Altheide  and  Johnson  refer  to  four  types  of  ‘interpretive  validity’  which 
can  be  used  to  judge  the  validity  of  qualitative  research:  usefulness,  contextual 
completeness,  research  positioning,  and  reporting  style.  Usefulness  involves  the  level  to 
which  the  study  “enlightens  those  who  read  it  or  moves  those  who  were  studied  to 
action”.  Contextual  completeness  deals  with  how  comprehensive  the  view  of  the 
situation  is  that  is  provided.  “Research  positioning  refers  to  researchers’  awareness  of 
their  own  influences  (both  subtle  and  direct)  in  the  research  setting.”  These  influences 
can  include  beliefs,  values,  and/or  biases.  Finally,  the  reporting  style  employed  by  the 
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researcher  can  have  an  effect  on  the  study’s  overall  credibility  (30: 168).  Steps  taken  to 
address  each  of  these  issues  are  described  below. 

With  respect  to  usefulness,  the  ultimate  detennination  can  only  be  made  some 
time  after  the  study.  However,  as  no  such  study  has  been  attempted  previously,  the 
results  will  inherently  be  enlightening  to  all  intended  audiences  (i.e.  the  ASC  Chief 
Architect,  DoDAF  Working  Group,  systems  engineering  community  in  general).  Even  if 
the  results  confirm  long-held  beliefs  as  to  the  level  of  implementation  of  the  architectural 
mindset,  this  study  provides  data  and  justification  for  those  beliefs.  In  terms  of  driving 
organizations  to  action,  this  study  includes  several  recommendations  in  order  to  address 
any  deficiencies  in  architectural  implementation  within  ASC.  Again,  the  final 
determination  will  be  made  in  how  many,  if  any,  of  the  recommendations  are  followed 
through. 

Altheide  and  Johnson  recommend  the  following  measures  to  address  contextual 
completeness:  “including  information  about  the  history  of  the  phenomenon;  the  physical 
setting;  the  activities,  schedules,  and  routines  of  the  participants;  as  well  as  their 
individual  perceptions  and  meanings”  (30: 168).  This  study  deals  with  the  issue  of 
contextual  completeness  by  capturing  the  evolution  of  the  DoDAF  and  its  expansion 
from  C4ISR  systems  to  all  weapon  systems  development  efforts.  Further,  the  physical 
setting  in  ASC,  the  activities  of  the  personnel  with  systems  architecture  responsibilities, 
and  their  perceptions  were  captured  as  part  of  the  data  collection  process. 

The  final  two  characteristics  of  validity  deal  with  the  researcher:  the  researcher’s 
positioning  and  reporting  style.  The  beliefs,  values,  and  biases  of  the  researcher  with 
respect  to  systems  architecting  and  the  DoDAF  are  presented  in  the  Limitations  section 
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of  this  chapter.  Full  disclosure  of  any  issues  that  may  affect  the  credibility  of  this  study 
is  the  goal.  In  terms  of  reporting  style,  this  study  is  presented  with  the  researcher’s 
interpretations  of  interviewee  views  as  expressed  in  the  interviews  conducted.  The 
overall  findings  and  conclusions  (i.e.  the  analysis)  are  based  on  these  interpretations; 
following  a  triangulation  methodology  involving  the  collection  of  like  statements  from 
several  interviewees,  “similar  themes  are  noted  in  data  collected  from  a  variety  of 
sources”  in  order  to  increase  credibility  in  the  interpretations  (30: 169).  Where 
appropriate,  to  bolster  the  weight  of  the  analysis,  representative  quotes  pertaining  to  the 
subject  matter  are  presented.  Finally,  outlier  analysis  examined  cases  that  differed 
markedly  from  the  majority  of  situations  investigated,  identifying  what  was  present  or 
absent  in  these  cases  compared  to  the  more  common  examples  (30: 169). 

In  addition  to  the  triangulation  method  described  above,  Cooper  identifies  a 
number  of  different  strategies  researchers  can  employ  to  increase  the  reliability  of  their 
research  designs  (1 1:6).  Specifically,  she  recommends  variety  in  data  collection,  which 
involves  collecting  data  from  a  number  of  different  locations  or  sources  (11: 12).  This 
technique  is  similar  to  the  triangulation  method  described  above  and  was  accomplished 
by  interviewing  personnel  with  systems  engineering  (and  presumably  then,  systems 
architecture)  responsibilities  at  various  organizations  within  ASC  (e.g.  Wings,  DRGs, 
XR,  CAG,  PM,  EN)  and  at  different  levels  within  these  organizations.  Further,  a  detailed 
literature  review  pertaining  to  systems  architecture  in  general,  the  DoDAF,  and  systems 
architectures  within  ASC,  in  particular,  provided  a  variety  of  additional  sources  of 
information. 
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The  limitations  of  this  study  are  common  to  qualitative  studies  employed  in  other 
disciplines  and  are  tightly  coupled  with  the  researcher’s  assumptions.  Specifically,  the 
key  limits  to  the  completeness  and  accuracy  of  this  study  are  access  to  the  right  people, 
the  researcher’s  positioning  with  respect  to  the  topic  of  study,  and  the  researcher’s  ability 
to  interpret  and  correctly  portray  the  true  state  of  systems  architecting  within  ASC.  In 
tenns  of  interviewee  selection,  the  primary  assumption  was  that  starting  with  the  Wing 
and  DRG  commanders  and  Directors  of  Engineering,  other  personnel  with  systems 
architecture  responsibilities  would  be  identified.  Although  this  occurred  in  many  cases, 
there  is  a  small  chance  someone  with  data  pertaining  to  this  study  was  not  identified  and 
therefore  not  interviewed.  With  respect  to  the  researcher’s  positioning,  one  assumption 
was  that  most,  if  not  all,  organizations  within  ASC  were  involved  in  at  least  some  level  of 
systems  architecture  work.  This  could  lead  to  a  limitation  in  terms  of  the  data  collected 
in  the  very  first  interviews.  Finally,  the  most  significant  potential  limitation  results  from 
not  having  another  research  cross  check  the  interview  data  in  order  to  ensure  the 
researcher’s  ability  to  correctly  analyze  the  data  collected  and  display  the  actual  state  of 
affairs  such  that  the  reader  has  the  same  picture  as  the  researcher. 

Assumptions  and  limitations  aside,  the  documented  methodology  suits  the  overall 
purpose  of  this  study  which  is  to  provide  a  status  of  systems  architectural  implementation 
within  ASC.  The  data  was  collected  through  numerous  interviews  and  grouped 
according  to  like  themes/beliefs.  These  groupings  facilitated  analysis  of  the  data  to 
determine  the  current  state  and  also  trends  in  systems  architecture  implementation  within 
ASC.  The  analysis  leads  to  the  derivation  of  significant  findings  and  conclusions.  This 
analysis  is  described  in  Chapter  4. 
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4.  Analysis  and  Findings 


4.1  Overview 

Chapter  3  outlined  the  data  collection  and  analysis  methodology.  Data  was 
collected  through  structured  interviews  intended  to  allow  the  identification  of  general 
groupings  of  architectural  implementation  and  overall  themes  within  ASC.  There  are  two 
types  of  analysis  techniques  at  work.  First,  there  is  the  logical  grouping  of  like 
organizations/groups  on  the  continuum  presented  in  Figure  7.  This  grouping  facilitated 
the  second  type:  descriptions  of  the  groups  themselves.  Finally,  the  analysis  leads  to  five 
findings  of  significance,  four  dealing  directly  with  the  research  objective  and  a  fifth 
having  ancillary  connection  with  the  topic. 

4.2  Logical  Grouping  Along  Implementation  Continuum 

As  outlined  in  the  previous  chapter,  this  continuum  provides  a  glance  into  the 
state  of  architectural  implementation  within  ASC.  Based  on  similarities  in 
responses/comments  from  interviewees,  organizations  were  placed  along  the  continuum 
indicating  whether  they  perfonned  “No  Architecture  Work”,  “Some  Architecture  Work”, 
or  a  “Great  Deal  of  Architecture  Work”.  The  placement  of  organizations  interviewed  is 
depicted  in  Figure  8  below.  The  characteristics  that  distinguish  each  group  are  discussed 
in  the  next  section. 
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Figure  8  ASC  Architecture  Implementation  Continuum 


It  is  interesting  to  note  that  the  organizations  grouped  on  the  left  side  of  the 
continuum,  the  “No  Architecture  Work  in  Organization”  section,  are  the  Wings  and 
Direct  Reporting  Groups  within  ASC.  However,  systems  within  many  of  these 
organizations  are  grouped  in  the  middle  of  the  continuum,  the  “Some  Architecture  Work 
in  Organization”  section.  These  groupings  are  highlighted  in  Figure  9  below. 
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Figure  9  Annotated  ASC  Architecture  Implementation  Continuum 


In  order  to  further  depict  this  phenomenon,  Figure  10  highlights  the  Long  Range  Strike 
Wing  position  on  the  continuum  -  specifically,  to  the  left,  while  three  of  the  systems  or 
sub-organizations  within  the  Wing  are  placed  toward  the  right  side  of  the  continuum. 
This  indicates  that  while  senior  leadership  within  an  organization  would  respond  that,  at 
least  “corporately”,  there  is  little-to-no  architecture  work  ongoing  within  the 
organization,  there  is  indeed  some,  and  in  at  least  this  particular  case,  a  great  deal  of 
architecture  work  in  progress.  This  phenomenon  is  discussed  further  in  the  Findings 
sections. 
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Figure  10  ASC  Corporate  Inconsistency  in  Implementation  Reporting 


4.3  Characteristics  of  the  Three  Groups  of  Organizations 

Within  each  of  the  three  groups  identified  in  Figure  8,  there  are  similar  beliefs 
and  behaviors  that  characterize  each.  These  similarities  involve  the  level  of 
understanding  of  the  value  of  systems  architectures  within  acquisition  programs,  the 
amount  of  exposure  in  terms  of  training  and  education  in  systems  architectures,  and  the 
DoDAF  in  particular,  and  the  different  regulatory  requirements  levied  upon  them. 

Further,  organizations/sy stems  grouped  within  each  category  also  often  shared  general 
acquisition  program  characteristics  such  as  location  in  the  acquisition  development  cycle 
(i.e.  pre-Milestone  A,  Milestone  B,  Sustainment,  etc.).  Each  grouping  is  described  below 
with  a  general  explanation  of  the  characteristics  of  the  organizations  in  the  group. 


50 


Organizations  in  the  “No  Architecture  Work”  Category 


Organizations  in  “No  Architecture  Work  in  Organization”  category  are  most 
generally  characterized  as  being  legacy  platforms  operating  under  existing  Operational 
Requirements  Documents,  as  opposed  to  the  newer  JCIDS  and  DoD  5000  series 
operating  instructions.  Organizations  such  as  the  Long  Range  Strike  Wing  with  the  B-l, 
B-2,  and  B-52,  the  Fighter  Attack  Wing  with  the  F-15  and  F-16,  the  Training  Group  with 
the  T-l,  T-6,  and  T-38,  and  the  Special  Operations  Forces  Group  with  their  C-130 
variants  all  have  systems  predominantly  in  sustainment  phase  of  development.  Further, 
these  programs  have  capability  “roadmaps”  in  lieu  of  integrated  architectures  to  address 
future  development  options.  Also,  the  personnel  assigned  have  virtually  no  training  or 
experience  with  the  DoDAF. 

Organizations  in  the  “Some  Architecture  Work”  Category 

Organizations  in  “No  Architecture  Work  in  Organization”  category  are 
characterized  by  new  acquisition  policy  driving  production  of  architecture  views, 
architecture  products  created  simply  as  an  output  of  good  systems  engineering  practice, 
and,  at  least  in  one  case,  a  Major  Command  (MAJCOM)  emphasis  on  integrated 
architectures.  These  organizations  create  architecture  products,  in  most  cases,  because 
they  are  required  to  manage  information  flows.  This  is  perhaps  an  indication  of  how 
architectures  could  become  a  part  of  the  weapon  systems  acquisition  process.  Program 
office  personnel  who  otherwise  may  not  have  taken  an  interest  in  the  DoDAF  will  gain 
exposure  because  of  necessity.  The  other  two  cases,  organizations  with  above  average 
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systems  engineering  practices  and  the  MAJCOM  driven  architecture  efforts  represent 
special  cases  within  this  group. 

In  terms  of  the  new  acquisition  policies,  JCIDS,  ISP,  and  NR-KPP  requirements 
have  lead  some  organizations  to  the  creation  of  architecture  products.  The  B-l  Fully 
Integrated  Data  Link,  B-2  Radar  Modernization  Program,  and  the  Personnel  Recovery 
Vehicle  programs  all  created  DoDAF  products  as  a  result  of  an  approaching  milestone 
decision  review.  In  this  instance,  the  views  were  created  by  in-house  personnel  (either 
government  employees  or  support  contractors)  with  limited  training  which  included  the 
DAU  Systems  Architectures  course,  SYS  283,  training  on  the  Popkin  System  Architect 
tool,  and  DoDAF  training  through  the  Air  Force  Chief  Architect’s  office. 

A  second  group  driven  by  regulatory  requirements  involves  programs  creating 
ISPs  and  the  need  to  address  NR-KPP  requirements.  The  F-16  Link  16  program  is  facing 
testing  through  the  Joint  Interoperability  Testing  Center,  which  requires  the  production  of 
an  ISP,  which  in  turn  requires  the  production  of  architecture  products.  Further,  the  C-17 
program  has  taken  an  even  stranger  trip  to  arrive  at  the  need  for  architecture  products. 

As  a  result  of  a  recent  Unit  Compliance  Inspection,  the  program  was  found  to  be  lacking 
a  Program  Protection  Plan  (PPP).  The  PPP  requires  an  ISP  as  an  annex.  And,  of  course, 
the  ISP  requires  several  DoDAF  views.  Although  one  respondent  stated,  “In  order  to 
meet  these  (documentation  requirements),  you  need  to  understand  the  architecture”,  the 
views  are  predominantly  created  by  contractors  as  the  government  personnel  had  no 
exposure  to  the  DoDAF.  It  is  interesting  to  note  that  the  programs  facing  a  milestone 
turned  to  in-house  personnel  to  create  the  architecture  products,  while  those  facing  ISP 
requirements  outsourced  the  creation  of  the  documents. 
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Two  special  cases  are  included  in  this  group:  organizations  that  create 
architectures  as  a  part  of  good  systems  engineering  practice,  and  organizations  working 
together  with  their  MAJCOM  on  integrated  architectures.  Organizations  in  the  first 
category  include  the  Global  Mobility  Wing,  the  Joint  Strike  Fighter  avionics 
organization,  and  the  F/A-22  program.  These  organizations  are  not  necessarily  driven  by 
JCIDS,  but  have  been  performing  systems  engineering  with  a  great  deal  of  rigor  and 
would  be  able  to  produce  DoDAF  products  simply  as  a  byproduct.  Finally,  the  other 
organization  in  this  category  enjoys  a  relationship  with  their  MAJCOM  where  the 
MAJCOM  emphasizes  integrated  architectures.  The  Air  Mobility  Command  drives 
Global  Mobility  Wing  efforts  through  architecture  products  created  in  their  A-6 
organization. 

Organizations  in  the  “ A  Great  Deal  of  Architecture  Work ”  Category 

Organizations  in  “Great  Deal  of  Architecture  Work  in  Organization”  category  are 
characterized  by  an  emphasis  on  future  systems/capabilities,  having  embraced  the 
benefits  of  systems  architecting,  and  having  personnel  with  significant  experience  in 
systems  engineering,  and  to  some  extent,  trained  in  the  DoDAF.  ASC  can  be  generally 
characterized  as  having  many  programs  that  have  been  in  development  for  a  long  time 
(F-15,  B-52,  C-130,  and  even  the  F/A-22).  However,  the  truly  new  capabilities  and 
systems  that  are  coming  into  ASC  for  development  include  the  Airborne  Electronic 
Attack  (AEA)  capability  and  the  Tanker  Modernization  Squadron.  These 
capabilities/systems  originate  in  ASC  with  the  ASC/XR  Capabilities  Planning 
organization.  ASC/XR  also  deploys  personnel  throughout  the  Wings  and  DRGs  as 
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liaisons  for  introducing  new  development  efforts.  Certainly  these  organizations  have  a 
regulatory  reason  to  perform  system  architecting,  however,  they  have  also  simply 
embraced  the  benefits  of  systems  architecting  to  acquisitions.  They  are  not  alone 
however,  as  the  B-2  Group  has  also  bought  into  the  positive  aspects  of  systems 
architecting  within  their  organization  -  and  further,  the  use  of  the  DoDAF.  All  of  the 
organizations  in  this  group  are  characterized  by  personnel  having  significant  experience 
in  systems  engineering,  and  to  some  extent,  trained  in  the  DoDAF. 

The  three  groupings  and  the  infonnation  underlying  the  placement  of  each 
organization  within  the  groups  lead  to  five  significant  findings.  The  first  four  relate 
directly  to  the  research  question  and  objective  -  what  is  the  state  of  DoDAF  system 
architecture  implementation  within  ASC.  The  last  finding,  although  not  directly 
answering  the  research  question,  is  closely  related;  dealing  with  the  transition  from 
system  oriented  to  capability-based  weapon  system  acquisition  processes  within  ASC. 
These  findings  are  described  below. 

4.4  Findings 

The  data  collection  and  analysis  process  lead  the  recognition  of  five  findings: 

1 .  ASC  organizations  are  not  doing  much  architecture  work  “corporately”. 

2.  If  the  organization  isn’t  developing  a  new  capability  or  doesn’t  have  an  ‘X’  in 

their  office  symbol,  they  are  not  likely  doing  any  architecture  work. 

3.  There  is  no  consensus  among  ASC  personnel  as  to  the  benefit  of  systems 

architectures  to  acquisition  program  management. 
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4.  The  leadership  and  organizations  attempting  to  instill  an  architectural  mindset 
in  ASC  programs  are  not  succeeding. 

5.  The  transformation  to  a  capabilities-based  weapon  system  acquisition  process 
is  not  complete. 

Each  finding  is  detailed  below  with  a  general  description  followed  by  representative 
comments  from  interviewees. 

Finding  One:  ASC  organizations  are  not  doing  much  architecture  work 
“ corporately 

Six  of  the  seven  Wings/DRGs  (or  85.7%)  are  only  doing  “Some”  or  “No” 
architecture  work.  There  is  no  high-level  acceptance  of  system  architectures,  as  indicated 
through  interviews  with  the  Commander’s  Program  Execution  Group  as  well  as  the 
senior  leaders  of  each  of  the  Wings  and  DRGs.  Although  the  senior  leaders  interviewed 
indicated  little  to  no  architecture  work  ongoing,  there  were  cases  where  organizations  or 
programs  within  the  corporate  organization  were  creating  architecture  products  due  to 
regulatory  or  other  requirements  (recall  the  Long  Range  Strike  Wing  example  in  Figure 
10).  Many  organizations  are  contracting  for  their  architecture  development  because  they 
felt  the  expertise/knowledge/experience  is  not  resident  in  ASC  organically. 

Comments  representative  of  this  finding  include: 

-  “The  Center  is  not  really  taking  advantage  of  architectures”. 

-  “Lack  of  architecting  work  has  a  lot  to  do  with  the  work  and  how  it  gets  to 
ASC”.  That  is,  in  platfonn  chunks  via  Program  Management  Directives  -  the  old 
acquisition  standby:  funding  and  requirements. 
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-  One  interviewee  identified  the  lack  of  implementation  as  essentially 
“pragmatism”,  believing  program  office  personnel  have  distinct  direction 
(funding  and  requirements  again),  which  are  difficult  enough  to  achieve  without 
the  additional  burden  of  systems  architecting. 

-  Most  of  systems  engineering  work  is  done  by  contractors. .  ,”we  have  no  one  left 
in  ASC  to  do  it”. 

Finding  Two:  If  the  organization  isn ’t  developing  a  new  capability  or  doesn ’t 
have  an  ‘X’  in  their  office  symbol,  they  are  not  likely  doing  any  architecture 
work. 

This  finding  involves  two  groups  -  those  with  an  ‘X’  in  their  office  symbol 
(ASC/XR  and  the  Wing/DG  XR  offices)  and  the  rest  of  ASC.  The  ‘rest’  of  ASC  is 
predominantly  described  above  in  Finding  One.  The  organizations  with  an  ‘X’  in  their 
office  symbol  are  focused  on  new  capabilities  and  systems  development.  However,  even 
within  this  group,  members  still  lack  the  training  and  experience  to  use  their  architecture 
products  as  a  systems  engineering  and  capability-based  analysis  decision-making  tool. 
Within  those  who  do  make  use  of  architectures,  there  are  three  groups.  The  first,  group 
includes  organizations  working  on  new  systems  and  either  required  to  under  JCIDS 
documentation  requirements  or  see  the  value  added  (XRs,  new  programs  like  AEA  and 
Tanker  Modernization).  The  second,  and  most  exclusive  group  is  those  who  create  the 
products  (or  more  appropriately,  could  create  the  products  if  required)  as  a  result  of 
rigorous  application  of  systems  engineering  practices  (Global  Recon  Wing,  etc.).  And 
finally,  the  largest  group  of  organizations  that  make  use  of  architectures  are  those 
directed  to  (programs/organizations  such  as  the  C-17,  B-2  RMP,  and  F-16  Link). 
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The  organizations  making  use  of  the  architectures  due  to  seeing  the  value  added 
are  described  in  greater  detail  later  in  this  chapter.  Specifically,  the  AEA,  Tanker 
Modernization  Squadron,  and  B-2  Group  are  highlighted  as  Success  Stories.  Details 
pertaining  to  these  organizations  are  included  later.  However,  below  are  representative 
comments  dealing  with  the  rationale  behind  the  organizations  without  the  ’X’  in  their 
office  symbols  are  not  using  architectures. 

-  “Can’t  very  well  change  the  architecture  of  a  legacy  system”. 

-  Bottom  Line:  “Putting  these  requirements  on  sustainment  systems  is  a  waste”. 

-  Program  managers  are  too  busy  putting  “rubber  on  the  ramp”. 

Finding  Three:  There  is  no  consensus  among  ASC personnel  as  to  the  benefit 

of  systems  architectures  to  acquisition  program  management. 

It  is  clear  from  the  respondents  there  is  no  consensus  as  to  the  value  of  systems 
architecting  within  acquisition  program  offices,  at  least  as  far  as  ASC  is  concerned. 
There  are  some  that  see  them  as  a  benefit  in  terms  of  a  tool  in  the  system  engineer’s 
toolbox.  While  others  believe  program  offices  are  not  the  real  users  of  the  architecture 
products.  Still  others  believe  the  products  and  process  are  too  complex.  Finally,  there 
are  many  who  believe  there  are  issues  with  the  products  themselves. 

For  those  who  see  system  architectures  as  a  benefit  in  terms  of  as  a  tool  in  the 
system  engineer’s  toolbox,  representative  comments  include: 

-  Architectures  are  a  good  communications  tool  to  HQ  (the  people  who  integrate 

systems). 

-  The  Systems  and  Technical  Views  are  what  provide  value  to  program  offices. 
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-  The  TV-1  is  an  important  product  that  establishes  the  standards  with  which  the 
rest  of  development  work  will  be  based  upon. 

-  These  products  would  really  benefit  complicated  commands  like  Air  Combat 
Command  and  Special  Operations  Command,  where  complex  interactions  and 
connectivity  are  the  norm. 

The  organizations  who  believe  program  offices  are  not  the  users  of  the 
architecture  products,  had  this  to  say: 

-  The  concept  (systems  architectures)  exists  at  an  Air  Force  level  such  that  it  has 
not  resulted  in  funding  and  requirements  outside  platforms.  In  other  words,  the 
program  offices  are  given  a  program  to  execute  and  the  architectures  should  be 
included  when  the  acquisition  effort  starts. 

-  A  lot  of  this  stuff  is  “too  high  level”. 

-  There  is  “No  advocacy  at  worker-bee  level  in  SPO  for  architectures”. 

-  Acquisition  personnel  need  good  examples;  a  “poster  child”. 

The  third  sub-finding  dealing  with  a  lack  of  consensus  within  ASC  revolves 
around  a  belief  that  the  products  and  process  of  systems  architecting  are  too  complex. 
Comments  in  support  of  this  belief  include: 

-  Architecture  products  are  “Complex,  time-consuming,  and  expensive”. 

-  They  are  “labor  and  intellect-intensive  matters”. 

-  There  is  a  danger  in  creating  something  so  “cumbersome”  that  it  is  too  abstract 
in  the  mainstream  of  the  management  of  the  program. 

And  the  final  sub-finding  in  this  area  deals  with  what  respondents  consider  issues 
with  the  DoDAF  products  themselves.  Representative  comments  include: 
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-  The  “views  are  not  well  defined”. 


-  “See  a  lot  of  people  running  around  creating  products  that  are  not  really 
integrated”. 

-  Many  products  lack  description  of  timing  and  latency. 

-  We  need  a  consistent  approach  for  the  implementation  of  the  views  into  the 
modeling  and  simulation  environments.  Specifically,  they  need  to  be  translated 
into  dynamic  system-of-system  operations  research  models  and  we  have  to  be 
able  to  tie  OVs  &  SVs  together  into  an  executable  models  that  are  good  for 
management  and  operational  assessments  (trade-offs,  impacts,  what  ifs,  etc.). 

Finding  Four:  The  leadership  and  organizations  attempting  to  instill  an 
architectural  mindset  are  not  succeeding. 

This  finding  deals  with  attempts  by  the  leadership  and  organizations  who  are 
attempting  to  instill  an  architectural  mindset.  According  to  the  interviewees,  these 
organizations  and  leaders  are  not  succeeding.  Comments  in  support  of  this  assessment 
include: 

-  “Death  by  viewgraph”  or,  “in  the  ether  frequently”.  There  is  a  belief  that  these 
organizations  are  really  good  at  putting  together  Power  Point  presentations,  but 
that  the  material  is  over  the  audiences  heads. 

-  With  respect  to  the  position  of  ASC  Chief  Architect;  most  interviewees  didn’t 
know  who  the  Chief  Architect  was  -  and,  in  some  cases,  that  ASC  even  had  one. 

-  The  ASC/XR  (Capabilities  Planning  Division)  plays  a  leading  role  in 
capabilities-based  analysis  and  planning  (with  architectural  pieces  to  both),  but 
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interviewees  believed  they  may  not  have  ability  to  do  what  we  need  to  have  done 
with  respect  to  systems  of  systems  development  planning;  specifically  the  trades, 
etc.  that  keep  the  product  center  from  getting  user-directed  solutions. 

-  Finally,  the  ASC  Program  Management  home  office,  ASC/PM  and  the  ASC 
Engineering  home  office,  ASC/EN,  have  responsibilities  to  train  and  equip  the 
program  managers  and  engineers  that  populate  the  program  offices.  With  respect 
to  systems  architectures,  there  is  no  fonnal  training  program  for  program 
managers  and  engineers. 

—  Engineers  can  volunteer  for  a  systems  engineering  certification  program 
through  the  Air  Force  Institute  of  Technology  that  includes  a  course  on 
systems  architecting.  However,  they  self-nominate  themselves  (i.e.  there 
is  no  selection  and  nomination  process),  there  are  no  positions  within  ASC 
that  are  identified  as  requiring  this  certification,  and,  at  least  in  the  past 
year  (2004),  there  were  only  four  engineers  participating  in  the  program. 

Finding  Five:  The  transformation  to  a  capabilities-based  weapon  system 
acquisition  process  is  not  complete. 

The  previous  four  findings  dealt  directly  with  the  research  topic  of  investigating 
the  state  of  systems  architecting  within  ASC.  This  last  finding,  although  no  less 
significant  in  terms  of  voracity  (as  it  was  apparent  from  the  number  of  interviewees  who 
shared  this  belief  that  it  was  worth  reporting),  does  not  tie  directly  into  the  research 
objective.  It  deals  with  the  adoption  of  a  capability-based  acquisition  process  within 
ASC.  This  shift  from  requirements-  and  platfonn-based  acquisition  is  fundamental  to  the 
new  JCIDS  process  and  therefore  subject  to  similar  growing  pains  in  terms  of  integration 
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into  the  standard  mode  of  operation  as  is  systems  architectures.  Indeed  the  two  concepts 
appear  to  be  following  similar  paths,  at  least  up  to  this  point  within  ASC.  Comments  that 
elucidate  this  finding  are  below. 

-  “I  have  a  user  that  doesn’t  know  what  a  capability  is  -  they  want  a  system”. 

-  Net-centricity  and  capabilities-based  development  has  not  found  its  way  into  the 
infrastructure. 

-  With  respect  to  the  notion  of  a  ‘Capability  Manager  (CM)’  where  the  CM 
allocates  requirement  to  system;  system  responsible  to  make  trade-offs  to  meet 
multiple  capabilities;  some  thought  that  this  was  a  dangerous  concept  if  they  are 
not  responsible  to  deliver  anything. 

-  Also,  there  was  concern  about  the  way  we  manage  capabilities/programs? 

—  Bottom-Up  (systems  perspective)  to  fill  capability  gaps,  or 
—  Top-Down  (capabilities-wise)  with  multiple  systems  to  meet  capability 
needs? 

These  findings  represent  the  complete  results  of  the  analysis  performed  of  the 
data  collected  during  this  study.  Four  of  the  findings  provide  direct  support  for 
answering  the  research  questions,  while  the  fifth  provides  useful  commentary  on  an 
ancillary  and  related  topic.  Although  the  tone  of  the  findings  presented  indicated  a  lack 
of  architecture  work  within  the  product  center,  there  are  some  who  are  making  it  work. 
Specifically,  the  AEA,  B-2  Group,  and  Tanker  Modernization  Squadrons  would  have  to 
be  considered  success  stories  in  terms  of  adopting  systems  architecture,  and  the  DoDAF 
mindset. 
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4.5  Success  Stories 


While  collecting  data  for  this  study,  three  organizations  stood  out  as  having 
successfully  applied  systems  architecture  within  their  sphere  of  weapon  system 
development.  These  organizations  arrived  at  their  acceptance  of  architectures,  and  the 
DoDAF,  in  different  manners,  but  all  represent  the  leading  edge  in  terms  of  architectural 
integration.  AEA  is  a  capability  that  is  being  managed  as  such.  The  Capability  Manager 
believes  systems  architectures  are  vital  to  the  success  of  a  capability  management  effort. 
The  B-2  Group  first  encountered  architectures  as  part  of  a  milestone  preparation  exercise 
for  the  Radar  Modernization  Program,  but  have  since  found  the  DoDAF  to  be  a  continued 
useful  tool.  Finally,  the  Tanker  Modernization  Squadron  also  started  off  without  the  aid 
of  system  architectures,  but  eventually  was  directed  to  take  a  capability  delivery 
approach.  Each  of  these  is  discussed  further. 

Airborne  Electronic  Attack  (AEA) 

As  mentioned  above,  AEA  is  not  a  single  program,  but  rather  a  capability.  At  this 
point,  the  organization  is  ‘prototyping’  the  Capability  Management  concept.  The  AEA 
Capability  Manager  (CM)  actually  has  his  own  Program  Element  which  allows  control  of 
the  funding  for  all  aspects  of  providing  this  capability.  The  CM  provides  funding  and 
requirements  to  the  appropriate  Wings/DRGs.  The  AEA  capability  is  intended  to  provide 
support  to  strike  forces,  initially  in  a  high  threat/limited  access  environment.  The  AEA 
architecture  contains  multiple  systems  including  the  B-52,  E-18,  and  the  Joint  Unmanned 
Combat  Aerial  System  (J-UCAS).  In  fact,  the  CM  believes  that  “architectures  are  key  to 
successful  capability  management”. 
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B-2  Group 


The  B-2  Group  within  the  Long  Range  Strike  Wing  took  a  unique  path  to  full 
systems  architecture  acceptance.  Initially  the  B-2  Group  faced  a  Milestone  B  decision 
for  the  B-2  Radar  Modernization  Program  (RMP).  Since  Milestone  B  requires  the 
creation  of  a  CDD,  the  program  office  was  required  to  create  architecture  views. 

However,  during  the  effort  of  creating  the  products  for  the  CDD,  the  effort  blossomed 
into  a  B-2  enterprise-wide  architecture  effort. 

The  program  office  personnel  realized  the  value  to  the  program  of  adopting 
architectural  methodology,  and  specifically  believed  it  would  be  faster  to  do  it  right 
(following  DoDAF)  than  to  fight  it  —  and  their  key  is  speed.  The  Group  sees 
architectures  as  a  “lingua  franca  to  go  from  operational  requirements  to  systems 
specifications”  and  maintain  infonnation  consistency.  They  hope  to  develop  a  consistent 
lexicon  of  speech  from  requirements  through  implementation.  Another  positive  aspect  of 
the  B-2  Group’s  architectural  implementation  is  their  cooperation  with  the  MAJCOM 
customer.  They  are  working  hand-in-hand  with  ACC/DRA2  (the  B-2  requirements 
office),  who  buys  in  to  the  value  of  architectures  as  well. 

Tanker  Modernization  Squadron 

The  Tanker  Modernization  Squadron  within  the  Global  Mobility  Wing  is  a  pre- 
Milestone  B  capability  development  organization  looking  into  replacing  the  aging  fleet  of 
KC-135  air  refueling  aircraft.  This  effort  began  originally  as  KC-767  Program  (the 
infamous)  Tanker  Lease  program.  At  the  early  stages,  any  architecture  work  completed 
by  the  people  in  the  program  was  essentially  “square-filling”.  However,  when  the 
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program  was  “paused”  for  political  reasons,  the  program  office  regrouped  and  began 
looking  at  the  requirement  from  a  capability  perspective.  They  were  initially  driven  to 
architectures  by  interoperability  and  infonnation  assurance  concerns,  but  have  embraced 
architectures  as  a  capability  development  tool.  Just  as  the  B-2  Group,  the  Tanker 
Modernization  Squadron  is  doing  it  right  -  working  with  the  MAJCOM  on  capability- 
based  development;  AMC  has  contracted  for  the  development  of  the  Tanker-X  CDD. 
Both  the  MAJCOM  and  the  Tanker  Modernization  Squadron  believe  this  early 
involvement,  coupled  with  the  use  of  systems  architectures,  provides  an  “opportunity  to 
get  on  top  of  requirements  generation  process.” 

These  three  organizations,  along  with  the  XR  organizations  represent  the  cutting 
edge  of  architectural  implementation  within  ASC.  They  represent  the  exception, 
however,  not  the  rule.  As  mentioned  earlier,  87%  of  the  Wings/DRGs  reported  only 
some  or  no  architecture  work.  The  data  collected  in  this  study  lead  to  the  additional 
findings  that  there  is  no  consensus  within  ASC  on  the  value  of  systems  architecting  and 
the  leadership  attempting  to  incorporate  architectures  into  ASC  acquisitions  is  not 
succeeding.  Further,  the  interviewees  indicated  an  additional  finding  concerning  the 
slow  pace  of  the  transfonnation  to  effective  capabilities-based  acquisition.  The  JCIDS 
process  is  intended  to  create  this  process,  and  systems  architectures  are  intended  to  aid  in 
the  execution  of  this  process.  Unfortunately  at  this  point,  despite  the  regulatory 
requirements  to  create  architecture  products,  the  program  managers  and  engineers  within 
ASC  are  not  reaping  the  intended  benefits  of  systems  architectures  within  their  programs. 
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5.  Conclusions  and  Recommendations 


5.1  Overview 

This  study  examined  the  level  to  which  ASC  has  implemented  systems 
architectures  within  their  weapon  systems  development  efforts.  The  literature  indicates 
there  are  real  benefits  to  be  had  for  acquisition  program  office  personnel.  In  addition, 
guidance  has  been  instituted  calling  for  the  use  of  system  architectures,  specifically  the 
DoDAF  products.  Despite  the  benefits  and  the  guidance,  there  are  roadblocks  to  the 
successful  implementation  within  ASC.  Based  on  a  robust  data  collection  and  analysis 
methodology,  five  critical  findings  were  identified.  These  findings  provide  the 
justification  for  the  two  conclusions  outlined  below. 

5.2  Conclusions 

Recall  the  specific  research  question  this  study  set  out  to  answer,  How  are 
systems  architectures,  and  the  DoDAF  specifically,  being  implemented  “in  the  trenches” 
of  USAF  weapons  systems  acquisitions?  This  study  focused  the  investigation  in 
answering  this  question  on  ASC.  The  data  collected  point  to  two  conclusions. 

1.  While  programs  required  to  follow  new  acquisition  processes  are  doing  so, 
very  few  are  employing  systems  architectures  systematically. 

2.  At  this  point,  at  least  within  ASC,  the  benefits  derived  from  an 
architectural  context  are  not  yet  being  realized. 

5.3  Recommendations 


65 


The  findings  provide  evidence  for  the  following  recommendations  offered  in 
satisfaction  of  the  research  objective.  There  are  recommendations  for  ASC,  for  the 
DoDAF  Working  Group,  and  for  the  systems  engineering  community  in  general. 
Ultimately,  these  groups  should  determine  the  goal  of  incorporating  systems  architectures 
within  weapon  systems  acquisitions  and  the  follow-up  with  a  level  of  commitment  to 
back  up  the  decision. 

1.  Lead  by  the  ASC  Chief  Architect,  ASC  leadership  should  clearly 
determine  goals  for  systems  architecting.  For  instance,  if  it  is  a  valuable  tool  that 
every  Wing  and  DRG  should  have  in  their  toolkit,  then  the  Chief  Architect  should  come 
up  with  architecture  standards  and  then  publicize/indoctrinate  the  program  managers  and 
engineers  within  the  center.  And  the  chief  engineers  within  each  Wing  and  DRG 
(spearheaded  by  ASC/EN)  should  take  lead  in  developing  architectural  mindset  within 
organization.  These  organizations  should  also  act  as  conduit  to  make  the  high-level 
(often  esoteric)  guidance  relevant  to  program  office  personnel  concerned  with 
cost/schedule/performance. 

2.  The  ASC  architectural  leadership  should  select  an  exemplar  case  as  a 
practical  example  of  how  systems  architectures  can  be  applied  to  capability 
management.  This  would  provide  other  program  offices  the  example  many  are 
clamoring  for.  Further,  a  more  comprehensive  training  program  should  be  instituted 
within  the  center  starting  at  the  “See  Dick  create  an  architecture”  level.  Several  training 
opportunities  already  exist:  AFIT  SENG  640  course,  DAU  SYS  283  course,  Aerospace 
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Corporation  Systems  Arch.  Course,  etc.  The  leadership  should  continue  the  current 
practice  of  using  ASC  Focus  Week  as  means  of  “getting  the  word  out”. 

3.  The  Architecture  Working  Group  (AWG)  should  incorporate  these 
findings  to  improve  DoDAF  Version  2.0  products  in  support  of  capabilities-based 
acquisition.  The  data  collected  and  presented  in  this  study  are  intended  to  aid  them  in 
making  the  DoDAF  more  of  a  tool  for  acquisition  program  office  personnel.  One 
imperative  is  for  the  AWG  to  make  the  case  for  architectures  as  systems  engineering  and 
program  management  tool.  Capture  the  examples  of  successful  implementation,  AEA/B- 
2/T anker  Replacement  within  ASC  -  but  also  any  others  from  other  product  centers  or 
services  for  that  matter,  as  case  studies  with  practical  examples  for  others  to  follow. 
Together  with  the  systems  engineering  community  in  general,  find  some  way  to  address 
issues  raised  in  Finding  #3  above  concerning  the  products  themselves;  timing  and 
latency,  how  to  address  complex,  dynamic  systems/capabilities  with  static  views. 

4.  The  systems  engineering  community  (SAF/AQ,  ASC/EN,  Chief  Engineers, 
CSE,  OSD/OSSE)  should  also  develop/integrate  a  syntax  and  methodology  for 
DoDAF  views  to  be  made  executable.  This  effort  should  be  more  than  just  to  determine 
if  the  architectures  will  work,  but  to  allow  for  the  development  of  measures  of 
performance  in  order  to  compare  architectures  to  one  another  for  tradeoff  purposes.  It  is 
this  ability,  to  dynamically  simulate  a  proposed  architecture  and  evaluate  the  measures  of 
effectiveness  output,  which  provides  the  ultimate  value  to  capability-based  weapon 
systems  acquisition  efforts. 
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5.4  Recommendations  for  Further  Study 

During  the  course  of  this  study,  several  additional  areas  of  interest  were 
identified,  where  further  study  could  be  of  value  to  decision-makers.  An  obvious 
extension  of  this  study  is  to  evaluate  the  other  Air  Force  product  centers  -  and  even 
investigate  the  state  of  architectural  implementation  within  the  other  services  acquisition 
programs.  As  previously  mentioned,  the  success  stories  within  ASC  should  be  studied  in 
greater  detail  in  order  to  produce  case  studies  that  could  serve  as  justification  for  the 
skeptical  and  a  notional  ‘how-to’  for  those  looking  for  an  example  to  follow.  Also  a 
recurring  theme  throughout  this  study  is  the  call  for  executable  architectures.  Efforts 
should  be  made  in  the  engineering  and  research  communities  to  develop  techniques  for 
the  dynamic  analysis  of  architectural  products.  Two  additional  recommended  areas  of 
study  are  capability  management  as  a  construct  and  the  Air  Force  Materiel  Command 
reorganization.  Both  areas  are  in  their  infancy,  in  that  they  are  recent  changes  to  the  way 
of  doing  business,  however  careful  study  as  these  concepts  mature  would  benefit  Air 
Force  acquisitions. 

The  first  recommendation,  in  tenns  of  additional  study,  is  to  reproduce  this  study 
at  other  product  centers  to  determine  if  ASC  is  out  front  of  the  systems  architecture 
implementation  curve.  How  are  the  other  centers,  Air  Armament  Center  (AAC), 
Electronic  Systems  Center  (ESC),  and  Space  and  Missile  Systems  Center  (SMC)  coming 
along  in  their  implementation  of  systems  architectures  within  acquisition  program 
management?  It  would  seem  notionally  that  ESC  would  be  in  front  due  to  their  exposure 
to  the  CAF  since  1996,  but  is  that  the  case?  Also,  SMC  follows  the  space-specific 
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guidance  of  the  NSS  03- 01,  which  calls  for  architecture  products  as  part  of  process;  but 
how  are  they  doing? 

The  acquisition  community  would  greatly  benefit  from  case  studies  documenting 
successful  lessons  learned  concerning  system  architectures.  Specifically,  an  AEA 
Capability  Management  Case  Study  would  have  value  no  matter  how  successful  this 
foray  into  capability  management  is.  If  it  works  and  the  capability  is  delivered  to  the 
warfighter  in  a  timely  and  cost-effective  manner,  this  could  be  a  model  for  other 
“capability  management”  scenarios  to  follow.  If  it  doesn’t  work,  then  the  focus  of  the 
case  study  could  be  “how  could  such  a  seemingly  good  idea  fail?”  In  either  case,  the 
DoD  is  moving  towards  a  capability-based  development  system,  and  as  the  Air  Force’s 
first  true  capability  management  effort,  AEA  bears  close  study. 

The  need  to  evaluate  methods  to  make  the  DoDAF  views  executable  has  been 
identified  repeatedly  in  this  study.  There  are  tools,  such  as  VITECH’s  CORE  that  allow 
the  system  engineer  to  “run”  the  architecture.  However,  at  this  point,  this  analysis  only 
provides  the  answer  to  the  question,  “does  the  architecture  modeled  work”?  This  is  truly 
an  important  question,  as  it  is  much  better  to  find  that  the  design  is  missing  key 
interactions  in  a  modeling  arena  as  opposed  to  when  you  have  built  hardware  and  are 
testing.  The  next  step,  however  is  the  ability  to  measure  how  well  each  architecture 
performs,  not  just  that  it  works. 

Closely  related  to  the  study  of  the  AEA  capability  management  effort  would  be  to 
further  explore  the  idea  of  Capability  Management  altogether.  For  instance,  what  exactly 
constitutes  capability  management?  What  differentiates  between  program  management 
and  capability  management  competencies?  Further,  who  are  capability  managers;  what 
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training/experience/knowledge  are  required?  Where  do  they  fit  organizationally  (product 
center,  AFMC  headquarters,  MAJCOMs,  USAF  CONOPs)?  What  about  funding  and 
control  of  schedules  to  synchronize  capability  releases?  This  is  an  area  ripe  for  study. 
Perhaps  investigating  how  similar  efforts  are  done  in  industry. 

The  final  recommendation  for  additional  study  would  be  to  evaluate  the 
effectiveness  of  AFMC  reorganization.  This  is  a  recent  change  to  how  AFMC  does 
business  and  was  done  to  accomplish  specific  goals.  The  study  could  be  a  case  study 
investigating  the  pros  and  cons  of  Wing/Group/Squadron  organization,  measuring 
progress  against  the  stated  goals  of  the  reorganization  effort.  Does  this  structure  really 
make  it  easier  to  relate  to  customer?  Does  the  new  structure  increase  cross-enterprise 
integration?  ASC  is  the  first  product  center  to  implement  this  reorganization,  but  the 
other  AFMC  product  centers  are  also  reorganizing  this  way. 

ASC  is,  as  part  of  the  overall  DoD  acquisition  transformation  effort,  in  a  period  of 
change.  The  aim  is  for  a  more  top-down,  capabilities-based  weapon  system  process 
where  all  services  developed  by  each  service  interact  to  produce  the  effects  combatant 
commanders  require  to  meet  the  national  security  objectives.  The  DoDAF  provides  a 
proposed  framework  for  the  development  of  architectural  products  in  support  of  the  kind 
of  analysis  required  to  make  this  vision  happen.  However,  at  least  at  this  point  in  ASC, 
the  architectures  have  not  been  integrated  into  the  everyday  operations  of  the  acquisition 
program  office  personnel  charged  with  managing  the  development  of  new  capabilities. 
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